Hands-free augmented reality system for picking and/or sorting assets

ABSTRACT

An asset identifier for an asset located within a physical environment is obtained based on received scanning information. The asset identifier is associated with a delivery destination defined for the asset. Asset location data that is associated with the obtained asset identifier and defines a position of the asset within the physical environment is generated based on the received scanning information. A current location of the physical environment is determined based on received asset location data. A projection device emits a navigational projection that corresponds to the position of the asset within the physical environment based on the asset location data and a determination that the determined current location is within a threshold distance of the defined delivery destination.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation-in-part of U.S. application Ser. No.16/103,566, entitled “Hands-Free Augmented Reality System For Pickingand/or Sorting Assets,” filed Aug. 14, 2018, which claims priority toU.S. Provisional Patent Application No. 62/545,752 entitled “Hands-FreeAugmented Reality System For Picking and/or Sorting Assets and Methodsof Utilizing The Same,” filed Aug. 15, 2017, all of which areincorporated herein by reference in their entirety.

This application also claims priority to U.S. Provisional PatentApplication No. 62/607,814, entitled “Hands-Free Augmented RealitySystem for Picking and/or Sorting Assets and Methods of Utilizing theSame,” filed Dec. 19, 2017, which is incorporated herein by reference inits entirety.

BACKGROUND

The automated handling of parcels (e.g., packages, containers, letters,items, pallets, etc.) transported by common carriers throughtransportation networks is a complex problem with many parts. No singlesystem or method alone appears to provide a comprehensive solution forall conditions. A primary component in some systems and methods forautomated handling of packages is a conveyance device (i.e., a conveyorbelt), which is generally formed and/or extended around at least twodriving wheels. Thus, by turning the driving wheels, the conveyor beltmay run continuously. Conveyor belts may also generally be flexible anddeformable at least while running in contact with the driving wheels,and a multitude of materials, linkages, and so forth have been used toachieve these goals.

Where automated handling of packages has been implemented, certaininefficiencies may arise. For example, where packages may be improperlyor too closely placed relative to one another on a conveyor belt,congestion may arise, impacting various measurements or the like thatneed to be performed on the packages while on the conveyor belt. Stillfurther, where the materials in which packages are wrapped (e.g., foilor paper or the like) differ in color or other material characteristics,inaccuracies may also arise in any measurements, imaging, orobservations made in an automated fashion relative to the packages orassets.

Beyond interactions with conveyor belts, automated handling of parcelscreates additional challenges related to how the parcels—referred toelsewhere herein as assets—are transported and/or handled by carrierpersonnel between conveyor belts (and the like) and respective inventorylocations (e.g., for picking) and/or sort locations (e.g., for sorting)associated with the assets.

In this context, a need exists for improved technological systems,assemblies, and/or methods for maintaining accurate records of thelocation of an asset in a sort and/or pick process, while also providingto carrier personnel improved instructions and/or guidance for theautomated handling of the packages within various environments (e.g., adelivery vehicle, a trailer or cargo area of a delivery vehicle, awarehouse environment whether relative to a sort location, a picklocation, a conveyor belt, and/or any combination thereof).

BRIEF SUMMARY

According to various embodiments described herein, a system is providedfor hands-free handling of at least one asset by a user. The system caninclude a user device configured to be worn by a user. The user devicemay include one or more memories and one or more processors configuredto perform the following operations. Asset identifier data can beobtained for at least one asset. Location data, associated with alocation for the at least one asset, can be determined based, at leastin part, upon the obtained asset identifier data. One or morenavigational projections configured to guide the user to the locationcan be dynamically generated and displayed. Handling of the at least oneasset by the user can be detected. One or more notifications associatedwith the handling of the at least one asset by the user at the locationmay be received.

In another embodiment, a computer implemented method is provided forhands-free handling of at least one asset by a user is provided. Themethod may include the following operations. Asset identifier data forat least one asset can be received from a remote location relative to auser device that is worn by the user. First location data associatedwith the user device can be determined at the user device. Secondlocation data associated with the at least one asset may be determined.The second location data may be determined, based at least in part, onanalyzing the received asset identifier data. The first location datamay be determined, based at least in part, on analyzing a presentposition of the user device. One or more navigational projectionsconfigured to guide the user to a location associated with the secondlocation data may be dynamically generated and displayed. The one ormore navigational projections may be dynamically updated based at leastin part on one or more detected changes of a present location of theuser device.

In yet another embodiment, a computer program product is provided forhands-free handling of at least one asset by a user. The computerprogram product may include at least one non-transitorycomputer-readable storage medium having computer-readable program codeportions stored therein. The computer-readable program code portions mayinclude one or more executable portions configured for performing thefollowing operations. An environment that a user is located in can bemapped based at least in part on generating a multidimensional graphicalrepresentation of the environment. Asset identifier data to identify atleast one asset can be received at a user device. One or more assetslocations can be associated within the mapped environment. The one ormore asset locations may be associated with the at least one asset. Oneor more navigational projections configured to guide the user to anasset location within the environment may be generated and displayedbased at least on the associating and within the environment that theuser is in.

In a further embodiment, a system is provided. The system comprises oneor more processors and one or more computer-storage media having one ormore instructions stored that, when used by the one or more processors,cause the one or more processors to perform the following operations.The operations comprise initializing a scanning device secured to astorage area of a delivery vehicle. Additionally, the operation comprisedetermining that an asset is located within the storage area based onscanning information obtained from the scanning device, wherein thescanning information includes an asset identifier associated with theasset and a defined delivery destination. The operations also compriseobtaining a current location of the delivery vehicle based on detectedlocation data. The operations further comprise, based on a determinationthat that the obtained current location is within a threshold distanceof the defined delivery destination, activating a projection device toemit a projection that corresponds to a determined position of theasset, the position of the asset being determined based at least in parton the obtained scanning information.

In another embodiment a computer-implemented method for scanning andlocating assets is provided. The method comprises obtaining, by acomputing device, an asset identifier for an asset located within aphysical environment based on received scanning information, wherein theasset identifier is associated with a delivery destination defined forthe asset. Additionally, the method comprises generating, by thecomputing device, asset location data that is associated with theobtained asset identifier and defines a position of the asset within thephysical environment based on the received scanning information. Themethod further comprises determining, by the computing device, a currentlocation of the physical environment based on received asset locationdata. The method also comprises causing, by the computing device, aprojection device to emit a navigational projection that corresponds tothe position of the asset within the physical environment based on theasset location data and a determination that the determined currentlocation is within a threshold distance of the defined deliverydestination.

In yet another embodiment, one or more computer-storage media havingcomputer-executable instructions embodied thereon that, when executed bya computing device, perform a method. The method comprises determining aphysical location of a delivery vehicle based on detected location data.The method also comprises obtaining an asset identifier for an assetstored within the delivery vehicle based on obtained scanninginformation, wherein the asset identifier is associated with a defineddelivery destination. The method further comprises determining alocation to emit a navigational projection within a cargo portion of thedelivery vehicle based on asset location data that is generated based atleast in part on the obtained scanning information. Additionally, themethod comprises causing a projection device secured to the cargoportion to emit the navigational projection directed to the determinedlocation based on a determination that the determined physical locationis within a threshold distance of the defined delivery destination.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING(S)

FIG. 1 schematically depicts a control system according to one or moreembodiments shown and described herein;

FIG. 2 schematically depicts a control system according to one or moreembodiments shown and described herein;

FIG. 3 schematically depicts a user device that communicates with acontrol system of according to one or more embodiments shown anddescribed herein;

FIG. 4 depicts a user device in conjunction with a harness mechanismaccording to one or more embodiments shown and described herein;

FIG. 5 depicts a user device in isolation without the harness mechanismaccording to one or more embodiments shown and described herein;

FIG. 6 schematically depicts a flowchart illustrating operations andprocesses performed by a user device according to one or moreembodiments shown and described herein;

FIG. 7 schematically depicts a flowchart illustrating operations andprocesses performed according to one or more embodiments shown anddescribed herein;

FIG. 8 depicts a facility and an environmental mapping procedureachieved via a user device according to one or more embodiments shownand described herein;

FIG. 9 depicts a facility and a pathway indicating navigationalprojection achieved via a user device according to one or moreembodiments shown and described herein;

FIG. 10 depicts a shelving containing portion of a facility and aplacement indicating navigational projection achieved via a user deviceaccording to one or more embodiments shown and described herein;

FIGS. 11A-C depict further views of three exemplary embodiments of theplacement indicating navigational projection achieved via a user deviceaccording to one or more embodiments shown and described herein;

FIG. 12 is a perspective or isometric view of a conveyor belt assemblythat may be utilized in conjunction with the control system and userdevice according to one or more embodiments shown and described herein;

FIGS. 13A-13F depict further views of additional exemplary navigationalprojections achieved via a user device and in conjunction with aconveyor belt assembly according to one or more embodiments shown anddescribed herein;

FIGS. 14A-B depicts a navigational projection within a physicalenvironment according to one or more embodiments shown and describedherein;

FIG. 15A is a perspective view of a physical environment according toone or more embodiments shown and described herein;

FIG. 15B is a side view of the physical environment of FIG. 15Aaccording to one or more embodiments shown and described herein;

FIG. 15C is a perspective view of a plurality of scanning devices havinga field of view of the physical environment of FIG. 15A according to oneor more embodiments shown and described herein;

FIG. 16 depicts an exemplary projector device according to one or moreembodiments shown and described herein;

FIG. 17 depicts a navigational projection within a physical environmentaccording to one or more embodiments shown and described herein; and

FIG. 18 is a flow diagram of an exemplary process for locating an assetaccording to one or more embodiments shown and described herein.

DETAILED DESCRIPTION OF VARIOUS EMBODIMENTS

The present disclosure will now be described more fully hereinafter withreference to the accompanying drawings, in which some, but not allembodiments of the disclosure are shown. Indeed, the disclosure may beembodied in many different forms and should not be construed as limitedto the embodiments set forth herein. Rather, these embodiments areprovided so that this disclosure will satisfy applicable legalrequirements. Like numbers refer to like elements throughout.

I. OVERVIEW

Existing asset handling technologies remain burdensome and are notsuitable for all tasks. For example, augmented reality-based computingsolutions have been pursued, such as with reference to U.S. Ser. No.15/581,609, the contents of which as are incorporated by referenceherein in their entirety. These augmented-reality-based solutions canutilize objects, such as smart glasses, to generate an environment so asto provide to carrier personnel (e.g., via a lens of the smart glasses)directions for transporting particular assets or packages. However,smart glasses may be uncomfortable to use for long periods of time(e.g., due to the weight and constant pressure) and these glasses reducethe peripheral vision for instructions needed for users or reduce visionin general due to glare on the lenses, which may impact both safety andjob accuracy. Existing technology solutions also fail to have morerobust functionality to meet the needs of multiple users for varioustasks. Further, some existing asset handling technologies are based onstatic projection methods that are also burdensome. For example, somesolutions require users to move around a large projector cart or wagonmounted with a generic projector to guide users to destinations.

Various embodiments of the present disclosure improve these existingtechnologies, such as smart glasses, by at least utilizing a hands-freeuser device(s), a control system or server in networked communicationwith the hands-free user device(s), and/or a generated augmented realityenvironment to facilitate handling and transport of an asset or packageby carrier personnel or the like. The handling and/or transport of theasset or package may be related to a picking of the asset from a picklocation (e.g., to “pull” the asset to fulfill an order thereof by acustomer), the sorting of the asset to a sort location (e.g., from aconveyor belt or the like to the next location in which transport orhandling of the asset may occur, for example, on the shelving of awarehouse or a vehicle). The hands-free user device(s) enables carrierpersonnel to transport and/or handle the asset or package in a safe,ergonomic, efficient, and accurate matter, regardless of where (e.g., toand from) the handling and/or transport is occurring, at least within athree dimensional environment mapped via the hands-free user device(s).

In an illustrative example of how these existing technologies areimproved according to aspects of the present disclosure, a user devicecan be worn by a user, such as on a wearable article of clothing, asopposed to placing eyewear over a user's eyes or using a mobile or cartdevice for the handling of assets. After one or more asset identifiersare obtained and location data is determined for an asset, one or morenavigational projections can be dynamically generated and displayed(e.g., within a physical environment a user is in, as opposed to a lensmedium) to guide the user to the location associated with the locationdata. Aspects can also detect handling of the asset by the user (e.g.,via cameras, sensors). One or more notifications associated with thedetection of the handling can be received (e.g., from a control system).Location data can be determined based on analyzing asset identifier dataand analyzing a present position of the user device. Further, a user'senvironment may be mapped based at least on generating amultidimensional graphical representation of the environment andassociating one or more asset locations within the mapped environment.At least each of these new functionalities improve existingtechnologies, as these are functionalities that various existingtechnologies do not now employ.

Conventional methods in the shipping and other industries rely uponcarrier personnel manually reading and/or scanning asset identifier dataassociated with the asset or package and then based thereon manuallytransporting the package or asset to the proper location. A pick orinventory location code or identifier and/or a sort location code oridentifier could then also—in some instances—be read or scanned toconfirm transport was correctly completed. Inefficiencies andinaccuracies are oftentimes encountered.

To address these inefficiencies and inaccuracies of the conventionalmethods, various non-conventional methods have been employed in thepresent disclosure. For example, such non-conventional methods includethe following operations: obtaining one or more asset identifiersdetermining location data for the associated asset(s). Navigationalprojections can be dynamically generated and displayed (e.g., within aphysical environment a user is in, as opposed to a lens medium) to guidethe user to the location associated with the location data. Aspects canalso detect handling of the asset by the user (e.g., via cameras,sensors). One or more notifications associated with the handling can bereceived (e.g., from a control system). Location data can be determinedbased on analyzing asset identifier data and analyzing a presentposition of the user device. Further, a user's environment may be mappedbased at least on generating a multidimensional graphical representationof the environment and associating one or more asset locations withinthe mapped environment. At least each of these new functionalitiesinclude non-conventional functions.

As used herein, an asset may be a parcel or group of parcels, a packageor group of packages, a box, a crate, a drum, a box strapped to apallet, and/or the like. According to standard practices, packages to besorted may be moved along a conveyor belt from some package source to anintake location (e.g., one or more sort employee workstations). A user(e.g., a sort employee or carrier personnel generally) may scan a barcode on the package, or simply reviews information printed on thepackage, and moves that package to an appropriate sort location (e.g., avehicle, a shelf, and/or the like) based on the information provided onthe package or via the barcode scanner. As described herein, embodimentsutilizing a conveyor belt assembly may rely upon an acquisition device(e.g., a stationary imager) positioned above the conveyor, upstream ofthe intake location or sort employee workstations to capture dataassociated with the package. Additional details in this respect may beunderstood with reference to U.S. Ser. No. 15/581,609, the contents ofwhich as are incorporated by reference herein in their entirety.

Via the hands-free user device(s), the carrier personnel or sortemployee may be guided to particular packages to select for transport.Upon the carrier personnel or sort employee picking up the particularpackages they are guided to, the hands-free user device(s) may beconfigured, according to various embodiments, to generate variousprojections, visible to the carrier personnel or sort employee. Thegenerated projections, which may be three-dimensional or two-dimensionalin form, are configured to guide the carrier personnel or sort employeefrom their current location to the appropriate sort location for theparticular package being handled. Upon arrival—via the guidance of thevarious projections—at the appropriate sort location, the hands-freeuser device(s), upon detecting a placement of the particular package mayfurther verify that the placement is correct. If incorrect,notification(s) may be generated, which notifications may take multipleforms, as detailed elsewhere herein.

In the context of picking, the control system may, via the network,interface with the hands-free user device(s) so as to generate one ormore of various projections to guide the carrier personnel or pickemployee to the location of a particular package that needs to be pickedor “pulled” for order fulfillment from a warehouse location or the like.Upon arrival of the carrier personnel or pick employee at the picklocation, specific projections may be generated, so as to advise thepersonnel or employee which specific package should be picked/pulledand/or how many packages (i.e., of the same type) should bepicked/pulled. It should be understood that upon or in response todetection of the picking/pulling of the package(s) by the user device,the latter may be further configured to then guide the carrier personnelto a subsequent location for ongoing handling/transport of the package.Exemplary subsequent locations may include a conveyor belt, a sortlocation, and/or a delivery location, as discussed above and alsodetailed elsewhere herein.

In certain embodiments, although not necessary via the three dimensionalmapping of the facility or warehouse and the network interface betweenthe user device(s) and the control system/server, the hands-free userdevice(s) may utilize software that not only detects changes in handlingof the packages (e.g., picking up or placement actions), but that alsodetects various markers or identifiers distributed throughout thefacility or warehouse, so as to ensure accuracy of the guidance and/ornavigational instructions provided to the carrier personnel. In otherembodiments, no such markers or identifiers may be provided, as thethree dimensional mapping via the user device(s)—with networkingconnectivity to the control system/server—may be utilized to calibrateand establish defined locations (i.e., pick or sort) throughout thefacility or warehouse prior to utilization of the hands-free userdevice(s) for operational purposes by the carrier personnel. In someembodiments, radio signal triangulation (RFID/WIFI), digital compass,and/or any other current method to determine indoor and outdoor positionand bearing may be utilized.

In the context of picking, the control system may also, via the networkor otherwise, interface with a fixed projector and/or the hands-freeuser device(s) so as to generate one or more of various projections toguide a delivery vehicle operator to the location of a particularpackage that needs to be picked or “pulled” for delivery at an addressat which the delivery vehicle is presently located. Upon entry of thevehicle operator (or other personnel) in the loading portion of thedelivery vehicle, specific projections may be generated, so as to advisethe personnel or employee which specific package should be picked/pulledfor delivery at the present address. It should be understood that upondetection of the picking/pulling of the package(s) by the projectorand/or the user device, at least the latter may be further configured tothen guide the carrier personnel or vehicle operator to a subsequentlocation for ongoing handling/transport of the package.

II. COMPUTER PROGRAM PRODUCTS, METHODS, AND COMPUTING ENTITIES

Embodiments of the present disclosure may be implemented in variousways, including as computer program products that comprise articles ofmanufacture. A computer program product may include a non-transitorycomputer-readable storage medium storing applications, programs, programmodules, scripts, source code, program code, object code, byte code,compiled code, interpreted code, machine code, executable instructions,and/or the like (also referred to herein as executable instructions,instructions for execution, computer program products, program code,and/or similar terms used herein interchangeably). Such non-transitorycomputer-readable storage media include all computer-readable media(including volatile and non-volatile media).

In one embodiment, a non-volatile computer-readable storage medium mayinclude a floppy disk, flexible disk, hard disk, solid-state storage(SSS) (e.g., a solid state drive (SSD), solid state card (SSC), solidstate module (SSM)), enterprise flash drive, magnetic tape, or any othernon-transitory magnetic medium, and/or the like. A non-volatilecomputer-readable storage medium may also include a punch card, papertape, optical mark sheet (or any other physical medium with patterns ofholes or other optically recognizable indicia), compact disc read onlymemory (CD-ROM), compact disc-rewritable (CD-RW), digital versatile disc(DVD), Blu-ray disc (BD), any other non-transitory optical medium,and/or the like. Such a non-volatile computer-readable storage mediummay also include read-only memory (ROM), programmable read-only memory(PROM), erasable programmable read-only memory (EPROM), electricallyerasable programmable read-only memory (EEPROM), flash memory (e.g.,Serial, NAND, NOR, and/or the like), multimedia memory cards (MMC),secure digital (SD) memory cards, SmartMedia cards, CompactFlash (CF)cards, Memory Sticks, and/or the like. Further, a non-volatilecomputer-readable storage medium may also include conductive-bridgingrandom access memory (CBRAM), phase-change random access memory (PRAM),ferroelectric random-access memory (FeRAM), non-volatile random-accessmemory (NVRAM), magnetoresistive random-access memory (MRAM), resistiverandom-access memory (RRAM), Silicon-Oxide-Nitride-Oxide-Silicon memory(SONOS), floating junction gate random access memory (FJG RAM),Millipede memory, racetrack memory, and/or the like.

In one embodiment, a volatile computer-readable storage medium mayinclude random access memory (RAM), dynamic random access memory (DRAM),static random access memory (SRAM), fast page mode dynamic random accessmemory (FPM DRAM), extended data-out dynamic random access memory (EDODRAM), synchronous dynamic random access memory (SDRAM), double datarate synchronous dynamic random access memory (DDR SDRAM), double datarate type two synchronous dynamic random access memory (DDR2 SDRAM),double data rate type three synchronous dynamic random access memory(DDR3 SDRAM), Rambus dynamic random access memory (RDRAM), TwinTransistor RAM (TTRAM), Thyristor RAM (T-RAM), Zero-capacitor (Z-RAM),Rambus in-line memory module (RIMM), dual in-line memory module (DIMM),single in-line memory module (SIMM), video random access memory (VRAM),cache memory (including various levels), flash memory, register memory,and/or the like. It will be appreciated that where embodiments aredescribed to use a computer-readable storage medium, other types ofcomputer-readable storage media may be substituted for or used inaddition to the computer-readable storage media described above.

As should be appreciated, various embodiments of the present disclosuremay also be implemented as methods, apparatus, systems, computingdevices, computing entities, and/or the like. As such, embodiments ofthe present disclosure may take the form of an apparatus, system,computing device, computing entity, and/or the like executinginstructions stored on a computer-readable storage medium to performcertain steps or operations. However, embodiments of the presentdisclosure may also take the form of an entirely hardware embodimentperforming certain steps or operations.

Embodiments of the present disclosure are described below with referenceto block diagrams and flowchart illustrations. Thus, it should beunderstood that each block of the block diagrams and flowchartillustrations may be implemented in the form of a computer programproduct, an entirely hardware embodiment, a combination of hardware andcomputer program products, and/or apparatus, systems, computing devices,computing entities, and/or the like carrying out instructions,operations, steps, and similar words used interchangeably (e.g., theexecutable instructions, instructions for execution, program code,and/or the like) on a computer-readable storage medium for execution.For example, retrieval, loading, and execution of code may be performedsequentially such that one instruction is retrieved, loaded, andexecuted at a time. In some exemplary embodiments, retrieval, loading,and/or execution may be performed in parallel such that multipleinstructions are retrieved, loaded, and/or executed together. Thus, suchembodiments can produce specifically-configured machines performing thesteps or operations specified in the block diagrams and flowchartillustrations. Accordingly, the block diagrams and flowchartillustrations support various combinations of embodiments for performingthe specified instructions, operations, or steps.

III. EXEMPLARY SYSTEM ARCHITECTURE

Generally, embodiments of the present disclosure relate to concepts forutilizing a hands-free user device(s), a control system or server innetworked communication with the hands-free user device(s), and agenerated augmented reality environment to facilitate handling andtransport of an asset or package by carrier personnel or the like. FIG.1 is a schematic diagram showing the exemplary communicationrelationships between components of various embodiments of the presentdisclosure. As shown in FIG. 1 , the system may include one or morecontrol systems 100, one or more user devices 110, one or more(optionally) location devices 415 associated with a location 400 (e.g.,a sort location or a pick location), one or more (optionally) conveyorbelt assemblies 800, and one or more networks 105. Each of thecomponents of the system may be in electronic communication with oneanother over the same or different wireless or wired networks including,for example, a wired or wireless Personal Area Network (PAN), Local AreaNetwork (LAN), Metropolitan Area Network (MAN), Wide Area Network (WAN),or the like. Additionally, while FIG. 1 illustrates certain systementities as separate, standalone entities, the various embodiments arenot limited to this particular architecture.

A. Exemplary Control System

FIG. 2 provides a schematic of a control system 100 according to oneembodiment of the present disclosure. As described above, the controlsystem 100 may be incorporated into a system as one or more componentsfor providing information regarding the appropriate location 400 foreach of one or more assets 10 (see FIGS. 8-12 ). In general, the termscomputing entity, computer, entity, device, system, and/or similar wordsused herein interchangeably may refer to, for example, one or morecomputers, computing entities, desktops, mobile phones, tablets,phablets, notebooks, laptops, distributed systems, gaming consoles(e.g., Xbox, Play Station, Wii), watches, glasses, key fobs, radiofrequency identification (RFID) tags, ear pieces, scanners, televisions,dongles, cameras, wristbands, kiosks, input terminals, servers or servernetworks, blades, gateways, switches, processing devices, processingentities, set-top boxes, relays, routers, network access points, basestations, the like, and/or any combination of devices or entitiesadapted to perform the functions, operations, and/or processes describedherein. Such functions, operations, and/or processes may include, forexample, transmitting, receiving, operating on, processing, displaying,storing, determining, creating/generating, monitoring, evaluating,comparing, and/or similar terms used herein interchangeably. In oneembodiment, these functions, operations, and/or processes can beperformed on data, content, information, and/or similar terms usedherein interchangeably. The control system 100 may also comprise variousother systems, such as an Address Matching System (AMS), an InternetMembership System (IMS), a Customer Profile System (CPS), a PackageCenter Information System (PCIS), a Customized Pickup and DeliverySystem (CPAD), a Web Content Management System (WCMS), a NotificationEmail System (NES), a Fraud Prevention System (FPS), and a variety ofother systems and their corresponding components.

As indicated, in one embodiment, the control system 100 may also includeone or more communications interfaces 220 for communicating with variouscomputing entities, such as by communicating data, content, information,and/or similar terms used herein interchangeably that can betransmitted, received, operated on, processed, displayed, stored, and/orthe like.

As shown in FIG. 2 , in one embodiment, the control system 100 mayinclude or be in communication with one or more processing elements 205(also referred to as processors, processing circuitry, and/or similarterms used herein interchangeably) that communicate with other elementswithin the control system 100 via a bus, for example. As will beunderstood, the processing element 205 may be embodied in a number ofdifferent ways. For example, the processing element 205 may be embodiedas one or more complex programmable logic devices (CPLDs),microprocessors, multi-core processors, co-processing entities,application-specific instruction-set processors (ASIPs),microcontrollers, and/or controllers. Further, the processing element205 may be embodied as one or more other processing devices orcircuitry. The term circuitry may refer to an entirely hardwareembodiment or a combination of hardware and computer program products.Thus, the processing element 205 may be embodied as integrated circuits,application specific integrated circuits (ASICs), field programmablegate arrays (FPGAs), programmable logic arrays (PLAs), hardwareaccelerators, other circuitry, and/or the like. As will therefore beunderstood, the processing element 205 may be configured for aparticular use or configured to execute instructions stored in volatileor non-volatile media or otherwise accessible to the processing element205. As such, whether configured by hardware or computer programproducts, or by a combination thereof, the processing element 205 may becapable of performing steps or operations according to embodiments ofthe present disclosure when configured accordingly.

In one embodiment, the control system 100 may further include or be incommunication with non-volatile media (also referred to as non-volatilestorage, memory, memory storage, memory circuitry and/or similar termsused herein interchangeably). In one embodiment, the non-volatilestorage or memory may include one or more non-volatile storage or memorymedia 210, including but not limited to hard disks, ROM, PROM, EPROM,EEPROM, flash memory, MMCs, SD memory cards, Memory Sticks, CBRAM, PRAM,FeRAM, NVRAM, MRAM, RRAM, SONOS, FJG RAM, Millipede memory, racetrackmemory, and/or the like. As will be recognized, the non-volatile storageor memory media may store databases, database instances, databasemanagement systems, data, applications, programs, program modules,scripts, source code, object code, byte code, compiled code, interpretedcode, machine code, executable instructions, and/or the like. Such codemay include an operating system, an acquisition module, a sort locationmodule, a matching module, and a notification module. The termsdatabase, database instance, database management system, and/or similarterms used herein interchangeably may refer to a structured collectionof records or data that is stored in a computer-readable storage medium,such as via a relational database, hierarchical database, and/or networkdatabase.

In one embodiment, the control system 100 may further include or be incommunication with volatile media (also referred to as volatile storage,memory, memory storage, memory circuitry and/or similar terms usedherein interchangeably). In one embodiment, the volatile storage ormemory may also include one or more volatile storage or memory media215, including but not limited to RAM, DRAM, SRAM, FPM DRAM, EDO DRAM,SDRAM, DDR SDRAM, DDR2 SDRAM, DDR3 SDRAM, RDRAM, TTRAM, T-RAM, Z-RAM,RIMM, DIMM, SIMM, VRAM, cache memory, register memory, and/or the like.As will be recognized, the volatile storage or memory media may be usedto store at least portions of the databases, database instances,database management systems, data, applications, programs, programmodules, scripts, source code, object code, byte code, compiled code,interpreted code, machine code, executable instructions, and/or the likebeing executed by, for example, the processing element 205. Thus, thedatabases, database instances, database management systems, data,applications, programs, program modules, scripts, source code, objectcode, byte code, compiled code, interpreted code, machine code,executable instructions, and/or the like may be used to control certainaspects of the operation of the control system 100 with the assistanceof the processing element 205 and operating system.

As indicated, in one embodiment, the control system 100 may also includeone or more communications interfaces 220 for communicating with variouscomputing entities, such as by communicating data, content, information,and/or similar terms used herein interchangeably that can betransmitted, received, operated on, processed, displayed, stored, and/orthe like. Such communication may be executed using a wired datatransmission protocol, such as fiber distributed data interface (FDDI),digital subscriber line (DSL), Ethernet, asynchronous transfer mode(ATM), frame relay, data over cable service interface specification(DOCSIS), or any other wired transmission protocol. Similarly, thecontrol system 100 may be configured to communicate via wirelessexternal communication networks using any of a variety of protocols,such as general packet radio service (GPRS), Universal MobileTelecommunications System (UMTS), Code Division Multiple Access 2000(CDMA2000), CDMA2000 1× (1×RTT), Wideband Code Division Multiple Access(WCDMA), Time Division-Synchronous Code Division Multiple Access(TD-SCDMA), Long Term Evolution (LTE), Evolved Universal TerrestrialRadio Access Network (E-UTRAN), Evolution-Data Optimized (EVDO), HighSpeed Packet Access (HSPA), High-Speed Downlink Packet Access (HSDPA),IEEE 802.11 (Wi-Fi), 802.16 (WiMAX), ultra-wideband (UWB), infrared (IR)protocols, near field communication (NFC) protocols, Bluetooth™protocols (e.g., Bluetooth™ Smart), wireless universal serial bus (USB)protocols, and/or any other wireless protocol.

The control system 100 may include or be in communication with one ormore input elements, such as a keyboard input, a mouse input, a touchscreen/display input, motion input, movement input, audio input,pointing device input, joystick input, keypad input, and/or the like.The control system 100 may also include or be in communication with oneor more output elements (not shown), such as audio output, video output,screen/display output, motion output, movement output, and/or the like.

As will be appreciated, one or more of the control system's 100components may be located remotely from other control system 100components, such as in a distributed system. Furthermore, one or more ofthe components may be combined and additional components performingfunctions described herein may be included in the control system 100.Thus, the control system 100 can be adapted to accommodate a variety ofneeds and circumstances. As will be recognized, these architectures anddescriptions are provided for exemplary purposes only and are notlimiting to the various embodiments. Additional details in this respectmay be understood from U.S. Ser. No. 15/390,109, the contents of whichas are incorporated herein by reference in their entirety.

As will also be appreciated, the control system 100 may be generallyconfigured to maintain and/or update a defined location map associatedwith a facility or warehouse in which the user device(s) will beoperated. This may be maintained for provision to the user device(s)upon calibration or initial “environment mapping” (see FIG. 8 ) via theuser device(s); in other embodiments, the control system may maintainthe defined location map—indicating where each package or asset shouldbe located (whether for picking or sorting)—as a fail-safe check orvalidation to be assessed against the environment mapping conducted viaand at the user device(s). In these embodiments, location devices 415(or identifier tags/codes/or the like) may be provided at the respectivelocations, for scanning or recognition via the user device(s) duringcalibration and/or environment mapping. In at least one preferredembodiment, however, the environment mapping occurs without need for orutilization of such location devices 415.

B. Exemplary User Device

FIG. 3 depicts a user device 110 that a user 5 (e.g., user 5 of FIGS.8-12 ) may operate. As used herein, a user may be an individual (e.g.,carrier personnel, such as a sort employee, a pick employee, or thelike), a group of individuals, and/or the like. In various embodiments,the user may operate the user device 110, which may include one or morecomponents that are functionally similar to those of the control system100. In one embodiment, the user device 110 may be one or more mobilephones, tablets, watches, glasses (e.g., Google Glass, HoloLens, VuzixM-100, SeeThru, Optinvent ORA-S, Epson Moverio BT-300, Epson MoverioBT-2000, ODG R-7, binocular Smart Glasses, monocular Smart Glasses, andthe like), wristbands, and the like, and/or any combination of devicesor entities adapted to perform the functions, operations, and/orprocesses described herein. In particularly preferred embodiments hereinthe user device 110 is a hands-free type device, including wearableitems/devices (e.g., the user devices of FIGS. 4-5 ), head-mounteddisplays (HMDs) (e.g., Oculus Rift, Sony HMZ-T3 W, and the like), andthe like. It should be understood that, in at least these embodiments,the user device 110 is configured to not impede the user's range ofvision and/or the like during the use thereof, as is possible in otherembodiments wherein, for example, the user device 110 is a type ofglasses or the like.

It should also be understood that the term user device 110 is intendedto refer to any device that projects, superimposes, overlays, orotherwise provides an image or projection on a surface with respect to auser's viewing angle or line of vision or a user device 110's angle.With reference now to FIGS. 4 and 5 , illustrated therein is anexemplary embodiment of a hands-free user device 110 according tovarious solutions described herein. The device 110 is configured forchest mounting via a mounting mechanism 112 in the illustratedembodiment. The mounting mechanism 112 may include a shoulder strapportion 112-1 configured to adjustably wrap around a user's anterior andposterior portion of the user's shoulders and a waist portion 112-2configured to adjustably wrap around the user's waist. The shoulderstrap portion 112-1 is connected to a top portion of the devicecomponent 114 at the anterior side and a top portion of the waistportion 112-2 at the posterior side. As discussed herein, it should beunderstood that the device 110 may come in any suitable form and bemounted in any suitable manner and in any suitable location. Forexample, in some embodiments, the user device 110 does not include themounting mechanism 112 and may be mounted to a component: on the user'shead (e.g., a hardhat), within a wristband, within a sock, within aglove, within a shirt, and/or within any other suitable article ofclothing in any orientation. It may also be provided with a mountingmechanism (not illustrated) on a picking cart or a forklift or any typeof component operated and/or being moved by the carrier personnel. Insome embodiments, other mounts could place the device 110 on top of thehead (e.g., via a helmet, cap or headband), or shoulder mounted. This,in order to avoid covering the projection element when carrying a parcelin front of the chest. In some embodiments, the projection element, thesensors and the processing units are mounted in different parts of thebody to get a better weight distribution.

Remaining with FIGS. 4 and 5 , the user device 110 in its hands-freeform may include not only the mounting mechanism 112 but also a devicecomponent 114 that together define and constitute the user device 110 insome embodiments. Via the device component 114 (FIG. 5 ), the userdevice 110 in its hands-free form may include an antenna 115 (e.g., theantenna 312 of the user device of FIG. 3 ), a camera 116, aspeaker/microphone 117, a pivoting laser projector 118 and two or morethree-dimensional depth sensors 119. In this respect, it should morebroadly be understood that the term user device 110 is intended to alsoinclude any other peripheral electronics and functionality that may beprovided in conjunction with such devices. For example, the user device110 may include speakers, headphones, or other electronic hardware foraudio output, a plurality of display devices, one or more positionsensors (e.g., gyroscopes, global positioning system receivers, and/oraccelerometers), battery packs, beacons for external sensors (e.g.,infrared lamps), or the like. In one embodiment, the user device 110 canbe used to provide an augmented reality environment/area, a mixedreality environment/area, and/or similar words, as may be used hereininterchangeably, to a user. The terms augmented/mixed environment/areashould be understood to refer to a combined environment/area includingthe physical environment/area and elements of a virtualenvironment/area. In some embodiments, the pivoting laser projector 118is alternatively an LED picoprojector.

In some embodiments, the device component 114 alternatively oradditionally includes different sensors for various functions, such asone or more digital compasses, accelerometers and/or gyroscopesconfigured to determine changes in position or speed of a user such thatthe pivoting laser projector 118 projects the correct image in thecorrect orientation. For example, if the user is hanging in a sidewaysmanner, an accelerometer can detect that the associated device component114 is likewise oriented. This information can be identified by aprocessor, which causes the pivoting laser projector 118 to responsivelytransmit a projection in a sideways manner, as opposed to a mannerassociated with the user standing on his/her feet. In another example,the user can be running or otherwise moving at a particular speed, whichcauses the projector 118 to make projections faster/slower based on thespeed or acceleration a user is moving at. Additionally oralternatively, these movement sensors can be used for notificationpurposes to the control system 100. For example, the accelerometer mayinfer that a person is in a particular orientation. These accelerometerreadings may then be transmitted, via the antenna 115, to the controlsystem 100 such that the control system 100 responsively transmits anotification back to the device component 114 in order to warn or notifythe user whether the user is in a suitable orientation. Other sensorsmay be used alternatively or additionally, such as range finders toidentify how far away the device component 114 is from obstacles (e.g.conveyor devices) within an environment. This may help the projectedimage be projected in a more precise three-dimensional manner. Forexample, referring to FIG. 12 , the projection 810 may be projected inits specific orientation based at least on one or more range findersidentifying the precise distance between a user device and the conveyingmechanism 802. In another example, proximity-based sensors (e.g., RFIDreader and tag) may be utilized within the user device and an object(e.g., an asset and/or conveying mechanism) in order to trigger theappropriate projections projected by the projector 118. For example, thedevice component 114 may include a tag reader that, when within aproximity or signal strength threshold of a tag located on anasset/object, triggers projections and/or notifications from the controlsystem 100. In another example, assets or other pieces of equipment(e.g., a conveyance mechanism) includes one or more beacons configuredto transmit location identifiers to any listening device within athreshold distance. In these embodiments, the listening device may bethe device component 114, which receives the location identifiers andtransmits them to the control system 100 such that the control system100 provides responsive notifications back to the device component 114,such as “pick package Y from shelf X,” etc. In some aspects, the devicecomponent 114 includes one or more location sensors (e.g., beacons, GPSmodules) for the determining and analyzing location data as describedherein.

Referring back to FIG. 3 , the user device 110 can include an antenna312 (e.g., the antenna 115 of FIG. 5 ), a transmitter 304 (e.g., radio),a receiver 306 (e.g., radio), and a processing element 308 (e.g., CPLDs,microprocessors, multi-core processors, co-processing entities, ASIPs,microcontrollers, and/or controllers) that provides signals to andreceives signals from the transmitter 304 and receiver 306,respectively. Certain embodiments of the user device 110 may alsoinclude and/or be associated with any of a variety of sensors (e.g.,three-dimensional sensors, such as the depth sensors 119 of FIG. 5 ),depth cameras (e.g., the camera 116 of FIG. 5 ), three-dimensionalscanners, binocular cameras, stereo-vision systems, pivoting projectors(e.g., the laser projector 118 of FIG. 5 ).

In certain embodiments, the three-dimensional sensors (e.g., sensors 119of FIG. 5 ) and/or the three-dimensional scanners may be utilized to“read” the environment surrounding the user device 110, as detailedelsewhere herein. In those and additional embodiments, the sensorsand/or scanners may build therefrom a three-dimensional model of thearea through which the device 110 travels and/or has travelled. Thisgenerated model, as detailed elsewhere herein, may then be compared byone or more processors within the device 110 to a memory-based map ofthe facility or area (i.e., the environment). By doing so, the scannerreadings may be used to determine which area of the map is in front of auser of the user device 110 (during operation) and extrapolate from thesame the position and heading of the user for future movement.

In certain embodiments, the pivoting projectors may be the pivotinglaser projector 118 of FIG. 5 ; although, in other embodiments theprojectors need not necessarily be laser-based. In various embodiments,it should be understood that the projectors are configured to generateand provide—in a manner visible to the user (e.g., the carrierpersonnel)—one or more navigational guidance projections (e.g., arrows,frames, text, and/or the like). These projections, as will be discussedin further detail elsewhere herein, may be two-dimensionalrepresentations (e.g., as illustrated in FIGS. 8-11C), three-dimensionalrepresentations (e.g., the projection 810 of FIG. 12 ), and/orholographic-based projections, or the like. The projections may beprovided on a floor surface, a wall surface, and/or on or adjacent ashelving structure, as detailed elsewhere herein. Optionally, a separatesensor, a regular camera, or the like (e.g., the camera 116 of FIG. 5 )may also be provided on the user device 110 for reading of the projectedimage(s) and therefrom verify if the projector is working properlyand/or whether the result projected is readable. Via the user device 110and software associated therewith, in conjunction with locationdetermining capabilities of the system described herein generally, theprojections provided via the user device are updated in real-time ornear-real-time, as the user moves physically. A refresh rate in therange of 35-60 times per second may be provided, although differingrates of refreshing may be desirable, provided that the rate provided issubstantially real-time or near real-time in nature.

Remaining with FIG. 5 , the camera 116 of the user device component 114illustrated therein in its hands-free form may be utilized as afail-safe for visual confirmation or the like of correct/accuratehandling of an asset or package by the user of the user device 110. Forexample, in some embodiments the camera 116 captures each assetidentifier and/or asset location identifier as a user traverses anenvironment. This location data and asset identifier data may then betransmitted, in near-real time via the antenna 115, to the controlsystem 100. The control system 100 may then compare the asset identifierto asset identifiers stored in a data store to identify an asset and dothe same with the captured location. In this way, the control system 100may identify any discrepancies between the asset and the location bylocating any mismatches between identifiers. For example, the controlsystem 100 may determine that the identifier associated with package Xshould be located, picked, and/or placed at shelf Y, but the camera 116captured it located in, picked, sorted and/or placed at shelf B. Anotification indicating this may be responsively transmitted back to thedevice 114 such that the speaker 117 issues a prompt indicating thediscrepancy and/or telling the user where the correct location is forthe particular package. In some embodiments, the device component 114itself determines this information without the need to transmit theinformation to the control system 100 for processing. In someembodiments, other notifications may be provided additionally oralternatively, such as a visual notification within a display screen onthe device component and/or a notification that causes vibration of thedevice component 114.

In certain embodiments, the camera 116 may be utilized as a verificationmechanism for ensuring that the projector 118 is working properly and/oris displaying readable projections. For example, the device component114 may stream in near-real-time information captured via the camera 116to the control system 100. If no projections are captured, this maytrigger an alert (e.g., to a supervisor mobile device), which indicatesthat the projections are not being made. Likewise, if a projection isnot verified (e.g., because there is a lot of light reducing projectionimage boundaries), a notification can be made in a similar manner asdescribed above. The speaker 117 may be utilized in conjunctiontherewith, so as to provide audible commands to the user (e.g.,delivered from the control system 100 to the component 114 via theantenna 115) should a deviation occur and/or to enable the user of theuser device to communicate, via the network, with the control system ina near real-time or real-time manner. For example, in some embodiments,the speaker 117 alternatively or additionally includes a microphone thatpicks up sound variations that are stored in the memory. The soundvariations may correspond to a command or natural language phrase issuedby the user, such as “where do I find item X?” or “where is shelf Ylocated?” Responsively, these sound variations are transmitted, via theantenna 115, to the control system 100. In these embodiments, thecontrol system 100 may employ one or more voice recognition algorithmsto interpret the sound variations and provide one or more responsivenotifications back to the device component 114, such that the speaker117 provides the notification output. For example, in response to theuser question of “where do I find item X?” the control system 100 mayinterpret the phrase, identify a data structure that associates thelocation with item X. The control system 100 may then responsivelytransmit to the device component 114 a notification that causes thespeaker 117 to output the location of where item X is.

Returning back to FIG. 3 , the signals provided to and received from thetransmitter 304 and the receiver 306, respectively, may includesignaling information in accordance with air interface standards ofapplicable wireless systems. In this regard, the user device 110 may becapable of operating with one or more air interface standards,communication protocols, modulation types, and access types. Moreparticularly, the user device 110 may operate in accordance with any ofa number of wireless communication standards and protocols, such asthose described above with regard to the control system 100. In aparticular embodiment, the user device 110 may operate in accordancewith multiple wireless communication standards and protocols, such asUMTS, CDMA2000, 1×RTT, WCDMA, TD-SCDMA, LTE, E-UTRAN, EVDO, HSPA, HSDPA,Wi-Fi, WiMAX, UWB, IR, NFC, Bluetooth™ Smart, USB, and/or the like.Similarly, the user device 110 may operate in accordance with multiplewired communication standards and protocols, such as those describedabove with regard to the control system 100 via a network interface 320.

Via these communication standards and protocols, the user device 110 cancommunicate with various other entities (e.g., the control system 100, alocation device 415, or the like) using concepts such as UnstructuredSupplementary Service Data (USSD), Short Message Service (SMS),Multimedia Messaging Service (MMS), Dual-Tone Multi-Frequency Signaling(DTMF), and/or Subscriber Identity Module Dialer (SIM dialer). The userdevice 110 can also download changes, add-ons, and updates, forinstance, to its firmware, software (e.g., including executableinstructions, applications, program modules), and operating system; thismay occur periodically, upon initiation via a user of the user device110, or upon cue(s) received at the user device from the control system100.

According to one embodiment, the user device 110 may also include alocation and/or perspective determining aspect, device, module,functionality, and/or similar words used herein interchangeably. Forexample, the user device 110 may include outdoor and/or environmentalpositioning aspects, such as a location module adapted to acquire, forexample, latitude, longitude, geocode, course, direction, heading,speed, universal time (UTC), date, and/or various otherinformation/data. In one embodiment, the location module can acquiredata, sometimes known as ephemeris data, by identifying the number ofsatellites in view and the relative positions of those satellites. Thesatellites may be a variety of different satellites, including Low EarthOrbit (LEO) satellite systems, Department of Defense (DOD) satellitesystems, the European Union Galileo positioning systems, the ChineseCompass navigation systems, Indian Regional Navigational satellitesystems, and/or the like. Alternatively, the location information may bedetermined by triangulating the user device 110's position in connectionwith a variety of other systems, including cellular towers, Wi-Fi accesspoints, and/or the like. Similarly, the user device 110 may includeindoor positioning aspects, such as a location/environment moduleadapted to acquire, for example, latitude, longitude, geocode, course,direction, heading, speed, time, date, and/or various otherinformation/data. Some of the indoor systems may use various position orlocation technologies including RFID tags, indoor beacons ortransmitters, Wi-Fi access points, cellular towers, nearby computingdevices (e.g., smartphones, laptops), nearby components with knownrelative locations, and/or the like. For instance, such technologies mayinclude the iBeacons, Gimbal proximity beacons, Bluetooth Low Energy(BLE) transmitters, Near Field Communication (NFC) transmitters,three-dimensional scanners, robot vision systems, environmental mappingdevices, and/or the like. These indoor positioning aspects can be usedin a variety of settings to determine the location of someone orsomething to within inches or centimeters.

The user device 110 may also detect markers and/or target objects. Forexample, the user device 110 may include readers, scanners, cameras,sensors, and/or the like for detecting when a marker and/or targetobject and/or a pattern of unique colors within its point-of-view(POV)/field-of-view (FOV) of the real world environment/area. Forexample, readers, scanners, cameras, sensors, and/or the like mayinclude RFID readers/interrogators to read RFID tags, scanners andcameras to capture visual patterns and/or codes (e.g., text, barcodes,character strings, Aztec Codes, MaxiCodes, information/data Matrices, QRCodes, electronic representations, and/or the like), and sensors todetect beacon signals transmitted from target objects or theenvironment/area in which target objects are located. For example, insome embodiments, the user device 110 may detect signals transmittedfrom the control system 100 (FIGS. 1-2 ), an asset 10 (FIG. 8 ), animproved conveyor belt assembly (FIG. 12 ), and/or from a locationdevice 415 (FIG. 1 ), as may be desirable or advantageous.

In one embodiment, the user device 110 may include accelerometercircuitry for detecting movement, pitch, bearing, orientation, and thelike of the user device 110. This information/data may be used todetermine which area of the augmented/mixed environment/area correspondsto the orientation/bearing of the user device 110 (e.g., x, y, and zaxes), so that the corresponding environment/area of the augmented/mixedenvironment/area may be displayed via the display along with a displayedimage. For example, the user device 110 may overlay an image in aportion of the user's POV/FOV of the real world environment/area. Inthese and other embodiments, the user device 110 may also includecircuitry and/or software for determining when a change in the handlingof a package or asset by a user of the user device has occurred.Exemplary changes detected may include the picking up of an asset orpackage, the setting down of an asset or package, or the like.

The user device 110 may also comprise or be associated with an assetindicia reader, device, module, functionality, and/or similar words usedherein interchangeably. For example, the user device 110 may include acamera or RFID tag reader configured to receive information from passiveRFID tags and/or from active RFID tags associated with an asset 10. Theuser device 110 may additionally or alternatively include an opticalreader configured for receiving information printed on an asset 10. Forexample, the optical reader may be configured to receive informationstored as a bar code, QR code, or other machine-readable code. Theoptical reader may be integral to the user device 110 and/or may be anexternal peripheral device in electronic communication with the userdevice 110. The optical reader may also or alternatively be configuredto receive information stored as human readable text, such ascharacters, character strings, symbols, and/or the like. The user device110 may utilize the asset indicia reader to receive informationregarding an asset 10 to be sorted.

In at least one embodiment, the user device 110 may be equipped with anoptical reader or the like configured to receive and/or monitorinformation associated with an associated conveyor belt, as detailedelsewhere herein. For example, the optical reader may be configured toreceive and/or otherwise monitor and/or recognize a pattern located onthe conveyor belt and associated with respective assets or packages.Additional details in this respect may be understood with reference toU.S. Ser. No. 15/581,609, the contents of which as are incorporated byreference herein in their entirety

The user device 110 may also comprise a user interface (that can includea display or see-through display 314 coupled to a processing element 308and/or a user input device 318 coupled to a processing element 308). Forexample, the user interface may be a user application, browser, userinterface, and/or similar words used herein interchangeably executing onand/or accessible via the user device 110 to interact with and/or causedisplay of information, as described herein. The user interface cancomprise any of a number of devices allowing the user device 110 toreceive data, such as a keypad (hard or soft), a touch display, voice ormotion interfaces, or other input device. In embodiments including akeypad, the keypad can include (or cause display of) the conventionalnumeric (0-9) and related keys (#, *), and other keys used for operatingthe user device 110 and may include a full set of alphabetic keys or setof keys that may be activated to provide a full set of alphanumerickeys. In addition to providing input, the user input interface can beused, for example, to activate or deactivate certain functions, such asscreen savers and/or sleep modes.

The user device 110 can also include volatile storage or memory 322and/or non-volatile storage or memory 324, which can be embedded and/ormay be removable. For example, the non-volatile memory may be ROM, PROM,EPROM, EEPROM, flash memory, MMCs, SD memory cards, Memory Sticks,CBRAM, PRAM, FeRAM, NVRAM, MRAM, RRAM, SONOS, FJG RAM, Millipede memory,racetrack memory, and/or the like. The volatile memory may be RAM, DRAM,SRAM, FPM DRAM, EDO DRAM, SDRAM, DDR SDRAM, DDR2 SDRAM, DDR3 SDRAM,RDRAM, TTRAM, T-RAM, Z-RAM, RIMM, DIMM, SIMM, VRAM, cache memory,register memory, and/or the like. The volatile and non-volatile storageor memory can store databases, database instances, database managementsystems, data, applications, programs, program modules, scripts, sourcecode, object code, byte code, compiled code, interpreted code, machinecode, executable instructions, and/or the like to implement thefunctions of the user device 110. As indicated, this may include a userapplication that is resident on the entity or accessible through abrowser or other user interface for communicating with the controlsystem 100 (FIG. 2 ), location device 415 (FIG. 1 ), and/or variousother computing entities.

In another embodiment, the user device 110 may include one or morecomponents or functionality that are the same or similar to those of thecontrol system 100, as described in greater detail above. As will berecognized, these architectures and descriptions are provided forexemplary purposes only and are not limiting to the various embodiments.

In the embodiment shown in FIGS. 4 and 5 , an information gatheringdevice may be provided via a combination of the image camera 116 that ismounted on the device component 114 and/or the three-dimensional depthsensors 119. In other embodiments, the information gathering device maybe a three-dimensional depth sensor, a stereo camera, and/or thelike—utilized independently relative to the camera. In all of these andadditional embodiments, the displayed or captured image data (e.g.,FIGS. 8-11 described elsewhere herein) is merged with objects in thephysical world/environment in a seamless manner, so as to provide asense that the displayed image(s) or projection is an extension of thereality present in the physical world/environment. This is oftentimesreferred to as a “mixed reality” or a “hybrid reality” environment,whereby the merging of real and virtual worlds produces a newenvironment containing visualizations of both physical and digitalobjects that are able to co-exist and interact relative to one anotherin a real-time manner. Stated otherwise, provided and/or generated is anoverlay of synthetic content on the real world or physical environment,with the former being anchored to and able to in a real-time manner(e.g., upon movement of a user) interact with the real world or physicalenvironment.

In the exemplary embodiment of FIGS. 4 and 5 , the overlay provided viathe user device 110 and its physical component 114 is at least a twodimensional representation that is projected ahead of or before the userof the user device, so as to provide handling/movement guidance (i.e.,navigational guidance) for the user during transport of or travel toinitiate transport of an asset or package.

C. Exemplary Conveyor Belt Assembly

FIG. 12 depicts a conveyor belt assembly 800 in communication with thecontrol system 100, where the improved conveyor belt assemblyfacilitates obtaining of asset 10 information. In the embodimentdepicted in FIG. 12 , the conveyor belt assembly 800 may comprise aconveying mechanism 802 and an acquisition/display entity 804 (forcapturing the asset 10 information), each of which as are described infurther detail in U.S. Ser. No. 15/581,609, the contents of which as areincorporated by reference herein in their entirety.

Of note relative to FIG. 12 , in conjunction with the hands-free userdevice(s) 110 described elsewhere herein, a user 5 may be guided towarda particular asset or package on the conveying mechanism 802 via one ormore navigational projections 810. In at least the illustratedembodiment, the projections 810 are provided in a three-dimensionalform, as may be compared with the two-dimensional projections in FIGS.8-11C. Either may be utilized interchangeably, as may beholographic-based projections or the like. Again, as mentioned,additional detail in this respect may be obtained from U.S. Ser. No.15/581,609, the contents of which as are incorporated by referenceherein in their entirety. In various embodiments, a hologram orholographic-based projection is a recording of a light field, as opposedto an image formed by a lens (e.g., of a pair of smart glasses), and isused to display a fully three or more dimensional image without the useor aid of special glasses or other intermediate objects. A hologram canbe displayed within any physical geographical environment without theneed of any projecting medium (e.g., projector screen, object, or lens).In some embodiments, the navigation projection 810 is or includes amultidimensional image that represents a volumetric display, which is avisual representation of an object in at least three physicaldimensions, as opposed to simulating depth or multiple dimensionsthrough visual effects. In these embodiments, the same projected object(e.g., an arrow pointing to an asset over a conveyor) looks differentfrom various perspectives (e.g., a side view, versus a front view,versus a back view). For example, a first front view can include a firstarrow and first instructions for worker X to pick/sort from. From asecond rear view, the same first arrow can include second instructionsfor worker Y to pick/sort from, etc.

D. Exemplary Location Device

In various embodiments, one or more locations 400 (and/or 1400) may beassociated with one or more (optionally provided) location devices 415,with both being configured for identifying one or more assets 10 beingsorted to each location 400. As non-limiting examples, such locations400 may include one or more vehicles (e.g., aircraft, tractor-trailer,cargo container, local delivery vehicles, and/or the like), pallets,identified areas within a building, bins, chutes, conveyor belts,shelves, and/or the like. The locations may be sort locations (fortransport of the asset for additional movement/handling) or picklocations (for storing of the asset until it needs to be picked or“pulled” for order fulfillment purposes of the like). The one or morelocation devices 415 (e.g., 415-1 of FIG. 13E) may be attached to alocation 400 and/or located more generally within and/or at the location1400 (see FIGS. 13A-13F). Alternatively the one or more location devices415 may be located adjacent to a sort location 400/1400 or otherwiseproximate the sort location 400/1400. In various embodiments, a locationdevice 415 may be located proximate to an area designated to store thesort location 400/1400. For example, when the sort location 400 includesa delivery vehicle, a location device 415 may be located above each of aplurality of parking areas designated for one or more delivery vehicles.This may apply equally relative to sort and/or pick locations (e.g.,FIGS. 13D-F).

In various embodiments, the one or more location devices 415 may includecomponents functionally similar to the control system 100 and/or theuser device 110. As noted above in referencing the control system 100,the term “computing entity” may refer to, for example, one or morecomputers, computing entities, desktops, mobile phones, tablets,phablets, notebooks, laptops, distributed systems, gaming consoles(e.g., Xbox, Play Station, Wii), watches, glasses, key fobs, RFID tags,ear pieces, scanners, televisions, dongles, cameras, wristbands, kiosks,input terminals, servers or server networks, blades, gateways, switches,processing devices, processing entities, set-top boxes, relays, routers,network access points, base stations, the like, and/or any combinationof devices or entities adapted to perform the functions, operations,and/or processes described herein Like the user device shownschematically in FIG. 3 , the location device 415 can include anantenna, a transmitter (e.g., radio), a receiver (e.g., radio), and aprocessing element (e.g., CPLDs, microprocessors, multi-core processors,co-processing entities, ASIPs, microcontrollers, and/or controllers)that provides signals to and receives signals from the transmitter andreceiver, respectively.

The signals provided to and received from the transmitter and thereceiver, respectively, may include signaling information in accordancewith air interface standards of applicable wireless systems. In thisregard, the location device 415 may be capable of operating with one ormore air interface standards, communication protocols, modulation types,and access types. More particularly, the location device 415 may operatein accordance with any of a number of wireless communication standardsand protocols, such as those described above with regard to the controlsystem 100. In a particular embodiment, the location device 415 mayoperate in accordance with multiple wireless communication standards andprotocols, such as UMTS, CDMA2000, 1×RTT, WCDMA, TD-SCDMA, LTE, E-UTRAN,EVDO, HSPA, HSDPA, Wi-Fi, WiMAX, UWB, IR, NFC, Bluetooth™, USB, and/orthe like. Similarly, the location device 415 may operate in accordancewith multiple wired communication standards and protocols, such as thosedescribed above with regard to the control system 100 via a networkinterface.

Via these communication standards and protocols, the location device 415can communicate with various other entities (e.g., the user device 110and/or the control system 100) using concepts such as USSD, SMS, MMS,DTMF, and/or SIM dialer. The location device 415 can also downloadchanges, add-ons, and updates, for instance, to its firmware, software(e.g., including executable instructions, applications, programmodules), and operating system. Of course, it should also be understood,as mentioned previously herein, that certain embodiments utilizing thehands-free form of the user device 110 may not utilize any locationdevices 415, whereby location of the various assets (for sorting orpicking) may be communicated directly as between the user device 110 andthe control system 100, further in conjunction with an environmentalmapping capability of the user device 110, as described further below.

E. Exemplary Location

Referring to FIGS. 11A-C and 13D-E, an exemplary location 400 (and/or1400) is schematically depicted. As described above, the location 400may include may include one or more vehicles (e.g., aircraft,tractor-trailer, cargo container, local delivery vehicles, and/or thelike), pallets, identified areas within a building, bins, chutes,conveyor belts, shelves, and/or the like. In the embodiment depicted inthe figures listed above, the location 400 includes a plurality ofshelves onto which the assets 10 may be placed and/or removed from.While these figures depict a specific quantity of shelves as beingstacked in a vertical direction, it should be understood that anyquantity of shelves may be arranged in any suitable configuration tohold the assets 10. Each of the shelves may include one or more visualindicators (e.g., 715 FIG. 10 ) positioned on or proximate to theshelves; however, as mentioned previously herein, certain hands-freeuser device embodiments may dispense with such indicators 110 c. Inthese and other embodiments, navigational projections 715, much like thenavigational projections 810 provided in conjunction with the conveyorbelt assembly 800 may assist in identifying an appropriate position forplacement and/or removal of the asset 10 within the sort and/or picklocation. In particular embodiments, for example, a user 5 (FIG. 12 )may utilize the indicia reader or camera of the user device 110 to scan,read, or otherwise receive asset identifier data from the asset 10 toidentify, in cooperation with the control system 100, an appropriateposition for placement of the asset 10 within the warehouse or facility,namely at the location 400/1400. In other embodiments, the controlsystem 100 may determine the appropriate position for placement of theasset within the warehouse relative to the location 400/1400 and conveythat information to the user device 110 in response to the user devicehaving approached an asset or package (e.g., for sorting). In otherembodiments, the control system 100 may proactively transmit projectionsto the user device 110 upon receipt of a package or asset order,requiring “picking” of the asset or package from a pick location fororder fulfillment purposes or the like. In still other embodiments, auser 5 may utilize the indicia reader or camera of the user device 110to scan, read, or otherwise receive asset identifier data from the asset10 at a location 400 that is associated with a mobile storage area(i.e., a delivery vehicle). In this and other embodiments, asillustrated in FIG. 14A, the delivery vehicle may be configured with aprojector 900 that proactively transmits navigational projections withinthe physical space of the storage area that is visible to the user,analogous to the user device 110. The navigational projections may bevisible within the physical space of the storage area so as to aid inthe selection (i.e., picking) of an asset or package. The projector 900may be used in these and other embodiments involving a delivery vehiclein conjunction with or in place of the user device 110. It should beappreciated that the location 400 can be mobile or static. In someaspects, the location 400 can be mobile as it may be associated with thedelivery vehicle. For example, the location 400 can be a cargo containerassociated with the delivery vehicle or within the delivery vehicleitself. In some aspects, the location 400 may be static. For example,the location 400 can be a storage area within a storefront. Additionallyor alternatively, the storage area can be behind a customer counter. Asa further example, the location 400 can be within a sorting facility.

Still further, the control system 100 may determine the appropriateposition for placement of the asset 10 within the location 400/1400based on a variety of factors. For example and without limitation, thecontrol system 100 may determine the appropriate position for placementof the asset 10 within the location 400 based on the destination of theassets 10.

F. Package Car Application

Turning now with particular focus upon FIGS. 14A-B, an exemplaryembodiment illustrates identifying an asset within a storage area of adelivery vehicle. It is contemplated that the term delivery vehicle maybe any kind of vehicle, such as an automobile, truck, train, orairplane. In this exemplary embodiment, the control system 100 may beconfigured to communicate with the delivery vehicle (i.e., location 400)in any of the ways and/or manners detailed elsewhere herein. The user 5may also, in exemplary embodiments, utilize a user device 110 in any ofthe ways and/or manners detailed elsewhere herein. In further exemplaryembodiments, the user 5 may dispense with the user device 110 and relyinstead upon instructions communicated via a projector 900 mounted onthe delivery vehicle (i.e., location 400). In some embodiments, theprojector 900 may be configured to communicate with the control system100 in a manner and/or way analogous to the communication between thecontrol system 100 and the user device 110, as detailed elsewhereherein. In some aspects, utilizing a projector 900 that is mountedwithin the storage area can be advantageous as it removes the need forthe user 5 to wear additional equipment. As the user 5 may be physicallyactive (e.g., carrying assets, entering and exiting the vehicle,climbing stairs at a delivery location), any additional equipment,including the user device 110, could interfere or encumber the user's 5movement. Additionally, due to the user's movement, smart glasses may beprone to falling off the wearer's head or obstructing the user's sight,presenting potential safety issues. In addition, an in-storage areamounted projector 900 can rely on a permanent power source. This is incontrast to a portable device that is powered by a portable battery,which may require constant recharging or replacement. As such, anin-storage area mounted system may be advantageous over a user device110, in some instances. It should be appreciated that while FIGS. 14A-18describe a location 400 with respect to the storage area associated witha delivery vehicle, the location 400 can be any physical environment.For example, the location 400 can be a physical environment within awarehouse, a sorting facility, a shopping area of a store, and the like.As such, the projector 900 can be secured within a physical environmentof a static location.

According specifically to the embodiment of FIG. 14A, in an analogousfashion as the “picking”-focused embodiments detailed elsewhere herein,the location of a set of packages (i.e. assets 10 a-d) may beilluminated by one or more navigational projections 901. It should beappreciated that, while not shown, the location of the set of packagesmay be highlighted by a user device (such as user device 110 of FIG. 4 )that generates one or more navigational projections. It should beunderstood that the one or more navigational projections 901, asillustrated in FIG. 14A, may be configured according to variousembodiments to operate and provide navigational guidance to a user 5 insubstantially the same way and/or manner as the one or more navigationalprojections 715 detailed elsewhere herein.

For purposes of background, the current process for drivers or personnelon a delivery vehicle (i.e., location 400) involves a manual sequencingof loading assets 10 inside the delivery vehicle. As the personnel goesto deliver the asset 10, he or she must sort through the assets todetermine the correct asset to deliver. Even though assets areconventionally loaded following a delivery sequence, the drivers orpersonnel must still spend some degree of time finding the right box ateach stop; oftentimes, additional boxes for a particular stop may beinadvertently overlooked. To alleviate and address the deficiencies ofthe current process's manual nature, the exemplary embodiment describedherein utilized augmented reality techniques (as described elsewhereherein) to highlight at each stop which asset(s) 10 is to be picked bythe drivers or personnel. This may be achieved, in certain embodiments,without utilization of smart glasses; alternative mechanisms include theuser device 110 (also as described elsewhere herein) and/or a projector900 mounted inside the delivery vehicle (i.e., location 400). As in thevarious embodiments described elsewhere herein, pattern recognition andmachine learning may be utilized for the control system 100 to, overtime, improve and understand how best to project and identify individualassets 10 and to build a three-dimensional representation of the assetsbased on two-dimensional images and/or three-dimensional sensor captureddata. In at least one embodiment, the projector 900 may be utilized toprovide navigational projections (e.g., “light” the asset) to theasset(s) 10 for picking at a particular service stop; this may be donein conjunction with—or in place of—a “lighting” of the asset(s) 10 viathe user device 110, as detailed elsewhere herein.

According to various embodiments the system may also, in addition to theprojector 900, have multiple components, including a scanning device1500 (e.g., a video camera, a three dimensional sensor or camera, aLIDAR scanner, tomography scanner, wireless RF signal scanner, or thelike). The scanning device 1500 can be used to capture the size andshape of asset 10 as it is loaded into the storage area and placed ontoshelves (i.e., specific locations). Any asset 10 added, stored, orremoved, can be captured by the scanning device 1500. The capturedinformation can then be processed via one or more computer processorsassociated with the scanning device, the control system 100, and/or theprojector 900. Pattern recognition, machine learning, and/or AI-basedalgorithms may be utilized to identify, from the scanning information,the shape, size, and position of each asset stored. Additionally oralternatively, the control system 100 may determine an asset identifierfrom the scanning information (which could be determined through imagerecognition that identifies a label on an asset and/or via wirelesssensors detecting an RFID signal, or the like). As described above, itshould be appreciated that while FIGS. 14A-18 describe a location 400with respect to the storage area of a delivery vehicle, the location 400can be any physical environment. For example, the location 400 can be astorage area within a warehouse, a sorting facility, a shopping area ofa store, and the like. As such, the scanning device 1500 can be securedwithin a storage area of a static location.

In some embodiments, the control system 100 can cause the scanningdevice 1500 to capture scanning information of a physical environment ofa storage area. Based on obtained scanning information, the controlsystem 100 can track the addition, movement, or removal of the asset 10from the storage area. For example, the asset 10 may be added to thestorage area. The addition of the asset 10 can be detected by thecontrol system 100 via one or more scanning devices 1500. As a furtherexample, the asset 10 may slide as a result of the delivery vehiclestopping, turning, or accelerating, or a user moving a particular parcelto a different location so as to reach a different parcel. The controlsystem 100 can cause scanning device 1500 can capture scanninginformation and then analyze the scanning information to determine thatthe asset 10 has moved to a new location. The control system 100 canstore the asset's location within a location database. As described inmore detail below, when the user 5 needs to pick the asset 10 when thedelivery vehicle arrives at a delivery location associated with adelivery destination of the asset 10, the projector 900 can providenavigational projections to guide the user 5 to the particular parcel.For example, the projector 900 may be configured to illuminate thetracked/monitored/identified asset 10, as detailed elsewhere herein.

Referring to FIG. 15A, one or more scanning devices 1500 can be used toobtain location data for an asset within an environment (e.g., a storagearea or a cargo area). As described herein, the scanning device 1500 maycomprise one or more sensors. For example, the scanning device 1500 caninclude a three dimensional sensor, an image sensor (e.g., video orcamera), a laser sensor (e.g., for use in LIDAR), infrared sensor,motion detector, and the like. The scanning device(s) 1500 can besecured to and/or mounted within the location 400 in any manner. Thescanning device 1500 can be mounted in a mobile or fixed fashion. Insome aspects, the scanning device 1500 can be a mobile 3D sensor thatcan sweep the location 400 to identify one or more assets. For example,the scanning device 1500 may be a component within the projector 900that rotates. It should be appreciated that the one or more scanningdevices 1500 may be secured along the ceiling, walls, or shelves withinthe storage area (e.g., location 400). It should be appreciated that theterm secured and/or mounted may refer to a permanent coupling or atemporary coupling such that the scanning device may be removed. Dataobtained by the scanning device can then be analyzed (e.g., by thecontrol system 100) to generate location data for an asset within thestorage area.

In some embodiments, the scanning device 1500 can be mounted to ashelving unit. The shelving unit can be positioned such that a face ofone shelf is opposite the face of another shelf. As shown in FIG. 15A,the face of a left shelving unit 410L is opposite the face of the rightshelving unit 410R. While only two shelving units are depicted, thestorage area may comprise rows of shelving units, as shown in FIG. 8 .

In some embodiments, one or more scanning devices 1500 can be mountedwithin the location 400 so as to have a field of view of a shelvingunit. As described above, the one or more scanning devices 1500 can bemounted along the ceiling, walls, or shelves within the storage area(e.g., location 400). In some aspects, the one or more scanning devices1500 can be mounted to a particular shelving unit so as to have a fieldof view of an opposing shelving unit. For example, the scanning device1500 can be coupled to a front face of the shelving unit. Additionallyor alternatively, the one or more scanning devices 1500 can bepositioned in series, along a front face of an individual shelf. Asshown in FIG. 15B, scanning devices 1500 can be positioned in seriesalong a plurality of shelves. For instance, the scanning devices 1500can be positioned in series along a first shelf 1530 and a second shelf1540.

In some embodiments, the sensors associated with each scanning device1500 have a field of view of an opposing shelving unit. As shown in FIG.15C, the series of scanning devices 1500 n can be positioned along theshelves of the left shelving unit 410L such that the field of view ofeach sensor produces a combined field of view 1520 of one or moreshelves of the right shelving unit 410R. While not shown, a secondseries of scanning devices 1500 a-f can be positioned along a rightshelving unit 410R to provide a combined field of view of the leftshelving unit 410L. Positioning the scanning device(s) 1500 along ashelf of an opposing shelving unit can be advantageous because itmaximizes the field of view of each scanning device 1500. This is incontrast to a wall or ceiling mounted scanning devices, which may havesensors that suffer from a limited field of view. For example, a ceilingmounted scanner might not be able to detect an asset that is positionedto the rear of the shelf. As shown in FIG. 15B, the series of scanningdevices 1500 a-f can be spaced apart horizontally (e.g., along thex-axis) and vertically (e.g., along the y-axis). Additionally, while thescanning devices 1500 n are illustrated as being aligned vertically orhorizontally, in some embodiments, the scanning devices are not alignedas such.

Utilizing data obtained from the scanning device 1500, the controlsystem 100 can generate location data for the asset 10 representing theasset's physical location within the location 400. The asset's locationdata can then be stored in an asset location database. The controlsystem 100 can update the asset location database based on determiningthat an asset has been added, moved, or removed from the storage area.In some aspects, the control system 100 can detect the addition (orremoval) of the asset based on analyzing data obtained via the scanningdevice 1500. For instance, the scanning device 1500 can detect (or nolonger detect) an asset identifier transmitted by an asset wirelesssignal generated by an RFID. Additionally or alternatively, the controlsystem 100 can determine that the scanning information no longerincludes a visual pattern (e.g., a QR code, particular dimensions of theasset) associated with a previously identified asset.

In some embodiments, to ensure that the scanning devices 1500 generateaccurate location data for the asset, the control system 100 candetermine whether there is an object (the user 5, a truck loader, etc.)in the storage area. The control system 100 can then determine that theobject may interfere with capturing scanning information. By way ofexample, the control system 100 can detect an object (e.g., the user 5)is in the aisle based on scanning information received via the scanningdevice(s), such as scanning information captured by an image sensor, amotion sensor, a thermal image sensor, or the like. Based on detectingan object in the aisle, the control system 100 can determine that theobject has interfered or will interfere with the scanning information.As such, the control system 100 can delay analyzing the scanninginformation so as to determine an asset's location. If the controlsystem 100 determines that no object is detected, the control system 100can analyze the obtained scanning information to generate asset locationdata. The asset location data can then be stored in an asset locationdatabase, which can be updated over time as new scanning information isobtained. In some embodiments, the control system 100 instructs thescanning device 1500 to receive scanning information based ondetermining that no object is detected.

In some embodiments, the control system 100 can generate asset locationdata based on position data for the scanning device. Among other things,the position data can define the physical location of the scanningdevice 1500 within the storage area. For example, referring to FIG. 15A,the control system 100 can store position data for each scanning device1500 a-f. In some aspects, the position data can include a measureddistance with respect to a point of origin. The control system 100 canutilize the point of origin as a reference to generate a value (e.g.,coordinates) for the asset location based on the location of thescanning device.

In some embodiments, the point of origin is associated with a physicallocation of a particular scanning device. For instance, the point oforigin can be associated with the physical location of scanning device1500 a. In some aspects, the position data for each scanning device 1500b-i can be determined with respect to the point of origin (e.g.,scanning device 1500 a). The position data can include a measureddistance (e.g., along the x-axis, y-axis, and z-axis) of each scanningdevice with respect to the point of origin. The control system 100 canthen utilize the point of origin to determine a value (e.g.,coordinates) for the asset's location.

By way of example, the control system 100 can obtain scanninginformation from scanning device 1500 b. The control system 100 can thenaccount for the position of the scanning device when analyzing thescanning information. For instance, the control system 100 can analyzethe scanning information received from scanning device 1500 b based onthe position of scanning device 1500 b. The control system 100 can thengenerate location data for the asset 10 from the scanning informationobtained for each the scanning device 1500 n. The asset location canthen be stored in an asset location database.

In some embodiments, the scanning device 1500 may comprise a wirelesssignal reader that can determine the location of the asset 10 throughwireless signals. By way of example, each asset 10 can be equipped witha tag (e.g., a microchip coupled to an antenna) that emits a wireless RFsignal that is received by one or more tag readers associated with thescanning device 1500. The wireless signal emitted by the tag can then beused to determine the location of the asset 10 within the location 400.For instance, as known in the art, a distance between the tag and a tagreader can be determined through relative signal strength intensity(RSSI) triangulation, Time Difference of Arrival (TDOA), and the like.

In some embodiments, the control system 100 can identify a particularasset based on asset characteristics captured by the scanning device1500. For example, the control system 100 can analyze the scanninginformation to identify a particular visual pattern or characteristicsassociated with the asset (e.g., a QR code, particular dimensions of theasset, or particular markings on the asset). Additionally oralternatively, the control system 100 can analyze the scanninginformation to identify a RF signal emitted from an RFID associated withthe asset. The control system 100 can then utilize this information toidentify the particular asset. It should be appreciated that theidentified asset can be associated with a unique identifier (e.g.,alphanumeric code). The control system 100 can then store the locationdata in association with the identified asset (e.g., associating thelocation data with the unique identifier). Additionally oralternatively, once the asset has been identified, the control system100 can determine a delivery location associated with the identifiedasset. For instance, the control system 100 can reference a databasethat comprises a delivery location for each identified asset.

In some embodiments, the control system 100 causes a projector 900 togenerate one or more navigational projections that identifies an assetto be pulled. As described herein, the one or more navigationalprojections provided by the projector 900 creates a visual cue for thephysical location of the asset 10. In other words, the projector 900 canilluminate a portion of the environment of the delivery vehicle so as toguide the user 5 to the particular asset.

In some embodiments, the projector 900 can include one or more lightsources mounted within the storage area (e.g., location 400). The one ormore light sources can be mounted at any location within the storagearea, including a ceiling, a wall, a floor, or a shelving unit. In someaspects, as shown in FIG. 14A and FIG. 16 , the projector 900 can be alight source that is mounted to the ceiling of the storage area. In someaspects, the one or more light sources can be mounted along the shelvesof the shelving unit 410 (e.g., along a front surface of a shelf). Itshould be appreciated that the projector 900 may include a light sourcethat projects a structured light in a particular direction. Additionallyor alternatively, the projector 900 is a light source that generatesomnidirectional light.

The light source can be a single, centralized light source or aplurality of distributed light sources. In some embodiments, the lightsource can be activated, by the control system 100, to illuminate aportion of the environment to identify the physical location of theasset to be pulled. Any light source may be used, including a halogenlight source, an LED light source, a laser light source, or the like.The light source can be a stationary light source or a rotatable lightsource. It should be appreciated that a stationary light source caneliminate the need for any moving parts, which can be beneficial in someinstances. For example, if the storage area is a portion of a deliveryvehicle, the movement of the vehicle along with the movement of arotatable light source can hinder an accurate placement of thenavigational projection. As such, a stationary light source may bepreferred. Still, in some instances, the rotatable light source may bepreferred. A rotatable light source can reduce installation time as itmay eliminate the installation of multiple, stationary light sources.

In some embodiments, the control system 100 can determine the locationof the navigational projection within the physical environment of thestorage area (e.g., location 400). For example, the control system 100can determine the location of the navigational projection based on theasset location data. The control system 100 can reference an assetlocation stored in the asset location database and cause a navigationalprojection to be presented proximate to the physical location of theenvironment that is associated with the asset location data.

In some embodiments, the control system 100 can activate the projector900 to generate a navigational projection in a particular portion withinthe physical environment. For example, the control system 100 can causea centralized light source to project light onto or near the surface ofthe asset. As a further example, the control system 100 can selectivelyactivate one or more light sources positioned near the asset. Asdescribed above, the projector 900 may comprise a plurality of lightsources that are distributed throughout the storage area. The controlsystem 100 can activate the projector 900 to generate a navigationalprojection in a particular portion of the environment by selectivelyactivating a light source among the plurality of light sources. Forinstance, based on determining the location of the asset 10 to bepulled, the control system 100 can selectively activate a particularlight source that is mounted proximate to the determined location of theasset 10. That is, based on the determined location of the asset, thecontrol system 100 can reference a physical location of the one or morelight sources and selectively illuminate a light source that is locatednear the generated asset location. It should be appreciated that thenavigational projection may illuminate an area within 0-20 feet of thesurface of the asset.

In some embodiments, the control system 100 can mechanically control thedirection of the light emitted by the projector 900. In some aspects,the control system 100 can generate a navigational projection in aparticular portion of the physical environment by mechanicallycontrolling the position of the projector 900. By way of example, asshown in FIG. 16 , the projector 900 may comprise a centralized lightsource 1600 that is rotatable, horizontally 1630 (e.g., about thex-axis) or vertically 1640 (e.g., about the y-axis). In some aspects,the light source 1600 can be rotated by one or more stepper motors 16101620. In some aspects, to ensure that the projector 900 projects a lightin the correct direction, one or more encoders (e.g., a light encoder ormagnetic encoder) can provide feedback to the control system 100 as tothe orientation of the rotatable centralized light source. Additionallyor alternatively, the rotatable centralized light source can berecalibrated or reset based on returning to an original position. Insome aspects, the control system 100 can control the direction of alight that is emitted by the projector 900 by controlling the positionof a mirror. For example, the control system 100 can cause a mirror torotate such that the mirror redirects a light emitted from a fixed,centralized light source toward a particular portion of the physicalenvironment, thereby indicating a particular asset to be pulled.

In some embodiments, as shown in FIG. 14 , the navigational projectionis an illumination of the asset 10. For example, the control system 100can cause a rotatable, centralized light source to shine a light onto asurface of the asset 10, thereby illuminating the asset 10. As a furtherexample, the control system 100 can selectively activate a stationarylight source to project a light onto a surface of the asset 10. A usercan then quickly identify the illuminated asset 10 to be pulled withouthaving to take the time to manually look through the storage area.

In some embodiments, the navigational projection can indicate that theasset 10 is behind a different asset. As shown in FIG. 17 , the asset 10to be pulled is positioned behind another asset 1700. In some aspects,the control system 100 can determine that the location of the asset 10is behind another asset 1700. For example, during loading of thedelivery vehicle, the control system 100 may determine, based onscanning information, that asset 1700 has been placed in front of theasset 10. For example, if the scanning device 1500 relies on imagesensor or depth sensors to determine the location of the asset, thecontrol system 100 can enter a loading state. During the loading state,the control system 100 can assume that any asset loaded will remainwithin the cargo area. The control system 100 can obtain first scanninginformation and identify the asset 10 has been loaded. The controlsystem 100 can then obtain updated scanning information and identifythat an asset 1700 has been placed in a similar location as asset 10.The control system 100 can then assume that asset 10 has been pushed torear of asset 1700. Additionally or alternatively, the control system100 can receive data from a user device indicating that asset 100 islocated behind asset 1700. In some embodiments, the control system 100can receive scanning information wirelessly and determine that the asset10 is located to the rear of asset 1700. Based on determining that asset10 is behind asset 1700, the control system 100 can then cause theprojector 900 to generate a distinct navigational projection orotherwise alter the navigational projection to indicate that the asset10 is located behind another asset 1700. For example, a lightilluminating asset 1700 may blink. As a further example, the distinctnavigational projection can be a particular color, symbol, or pattern toindicate that the user should look behind the asset 1700 to find theasset 10 to be pulled.

In certain embodiments, either upon request from the driver or personnelor automatically based upon—as a non-limiting example—GPS positionaldata associated with the delivery vehicle (i.e., location), the controlsystem 100 will communicate—to the projector 900 and/or the user device110—instructions identifying which of one or more assets 10 should behighlighted as the driver or user 5 enters the package storing portionof the delivery vehicle. As described herein, in some embodiments, assetlocations can be known based upon the three-dimensional representationof the assets 10 mapped via the control system 100, with capabilitiesincluding mapping of shelving, floor, and/or aisle locations for variousassets 10.

In some embodiments, the control system 100 can monitor the location ofthe delivery vehicle. For example, the control system 100 can obtainlocation data from one or more location modules. The control system 100can then utilize the location data to indicate the physical location ofa delivery vehicle. It should be appreciated that the location module(e.g., a GPS location module) can be a component of the control system100, the delivery vehicle, or a user device. Based on obtained locationdata from the location module, the control system 100 can determine thevehicle's proximity to a delivery destination of a particular assetstored in the delivery vehicle. The control system 100 can determine thedelivery destination for any particular asset from a databaseassociating asset identifier of the particular asset with it respectivedelivery destination. The control system 100 can thus receive an assetidentifier associated with each asset being transported and determinethe delivery destination of each asset.

Continuing, in some embodiments, the control system 100 can determinewhether a detected location is associated with a delivery location of anasset. For instance, the control system 100 can determine whether thedelivery vehicle is within a predefined threshold of a delivery locationassociated with an asset. The predefined threshold can be any thresholddistance, including a foot up to several miles. If the deliveryvehicle's location is within a predefined threshold, the control system100 can determine the one or more assets to be pulled. The controlsystem 100 can then direct the user 5 to the asset 10 to be pulled fordelivery to a location (e.g., house, apartment, building, or smartlocker) proximate the vehicle's physical location.

In some embodiments, the control system 100 can cause one or morenavigational projections to be generated based on predefine conditions.In some aspects, the predefined conditions may be associated with thestate of the delivery vehicle. For example, the control system 100 candetermine whether the delivery vehicle's gear has been placed in park.If so, the control system 100 can automatically cause the projector 900to generate the one or more navigational projections. As anotherexample, the control system 100 can determine that the deliveryvehicle's door has been opened (e.g., the opening of a door to thestorage area or driver side door). In some aspects, the predefinedcondition may be associated with receiving a command signal. Forexample, the user 5 may arrive at a particular stop and activate acommand signal (e.g., through a user device or a switch mounted withinthe delivery vehicle). Based on receiving the command signal, thecontrol system 100 can cause one or more navigational projections to begenerated.

In various embodiments, the control system 100 can exchangeasset-related data with a user device regarding the handling of theasset. For example, the control system 100 receive asset-related datafrom a user device (a handheld computing device or user device 110)indicating that a particular asset has been loaded or unloaded. In someaspects, the asset-related data can include an asset identifier,dimensions, a weight, or a delivery destination. The control system 100can receive asset-related data during the loading of the asset, whichcan then be used to determine the location of the asset. For example,the control system 100 can analyze scanning information for an assethaving a particular asset identifier or having particular dimensions. Asa further example, the control system 100 can receive asset-related datarelated to the unloading of the asset from the cargo area. The controlsystem 100 can utilize the asset-related data to determine that theasset has been removed and that the asset will be absent from anyfurther scanning information.

Turning now to FIG. 18 , an exemplary flow diagram 1800 shows a processof locating an asset. At Block 1810, a scanning device is initialized.For instance, the control system 100 can instruct a scanning device 1500to begin obtaining scanning information regarding a cargo container(e.g., location 400) of a delivery vehicle. In some embodiments, thecontrol system 100 can instruct a plurality of scanning devices toobtain information regarding the cargo area. In some aspects, thecontrol system 100 can initialize the scanning device based ondetermining that no object will interfere or disrupt the scanninginformation obtained by the scanning device. The scanning device canthen capture scanning information. For instance, as described herein,the scanning device 1500 can receive scanning information through imagesensors, depth sensors, wireless sensors, and the like. It should beappreciated that while the exemplary flow diagram 1800 refers to a cargocontainer of a delivery vehicle, the steps could be performed withrespect to any location.

At Block 1820, it can be determined that an asset is located within acargo container. In some embodiments, the control system 100 candetermine that an asset is located within the cargo container (e.g.,location 400) based on scanning information obtained from a scanningdevice (e.g., scanning device 1500). It should be appreciated that thecontrol system 100 can receive, from a scanner (e.g., scanning device1500 or handheld scanner), scanning information including an assetidentifier associated with the asset and a defined delivery location. Insome aspects, the control system 100 can determine an asset identifierbased on analyzing the scanning information. For instance, the controlsystem 100 can analyze the scanning information for distinguishingcharacteristics of the asset, such as a distinct visual aspectassociated with the asset (such as an alphanumeric code, QR code,dimensions of the asset, symbols, and the like) or a wireless signal(e.g., an RFID signal communicating an asset identifier). Based onscanning information, the control system 100 can determine that aparticular asset is located within the cargo container. As describedherein, the control system 100 can also determine a particular locationfor the particular asset. It should be appreciated that the controlsystem 100 can reference a database linking the asset identifier and theparticular delivery location so as to determine a delivery destinationfor the asset.

At Block 1830, a current location of the delivery vehicle is obtained.For example, the control system 100 can utilize location data that isdetected from one or more location modules. The location module (e.g., aGPS location module) can be a component of the control system 100, thedelivery vehicle, or a user device.

At Block 1840, a projection device can be activated. For example, basedon a control system 100 determining that a current location of thedelivery vehicle is within a predefined range or threshold of a deliverydestination, the control system 100 can activate a projection device 900to emit a projection that corresponds to a determined position of theasset. As described herein, the position of the asset can be determinedbased on the scanning information obtained from the scanning device1500. In some embodiments, the scanning information is a scannedposition of the asset within the storage area. For example, the scanningdevice 1500 can detect a particular position of the asset relative tothe scanning device 1500 based on one or more sensors, such as a depthsensor, an image sensor, a wireless signal sensor, and the like.

In some aspects, the projection is emitted or generated by a lightsource associated with the projection device 900. For example, thecontrol system 100 can activate the projector 900 to generate anavigational projection in a particular portion of the environment byselectively activating a light source among the plurality of lightsources. For instance, based on determining the location of the asset 10to be pulled, the control system 100 can selectively activate aparticular light source that is associated with the determined locationof the asset 10. That is, based on the determined location of the asset10, the control system 100 can reference a physical location of the oneor more light sources and selectively illuminate a light source that islocated near the determined asset location. As a further example, thecontrol system 100 can cause a centralized light source to project lightonto or near the surface of the asset. It is contemplated that theprojected light can be within 0-20 feet of the surface of the asset.

In some embodiments, the control system modifies projection coordinatesof the projection device based on the scanned position and thedetermination that the current location is within the threshold distanceof the defined delivery destination. For example, the control system 100can determine a placement of a projection within the storage area fromscanning information receiving from scanning device 1500. The controlsystem 100 can then cause a centralized light source of the projector900 to rotate so point in a particular direction. The control system canthen instruct the projector 900 to emit the projection based on themodified projection coordinates.

In some embodiments, additional scanning information is obtained. Forexample, the control system 100 can instruct the scanning device 1500 toobtain additional scanning information. In some aspects, the controlsystem 100 can instruct the scanning device 1500 to obtain additionalscanning information based predetermined conditions, such as based ondetermining a vehicle door has been closed, determining that the vehicleis moving or has stopped, determining that a particular time intervalhas lapsed, and the like. The additional scanning information caninclude an updated scanned position of the asset within the storagearea. This scanning information can then be analyzed by the controlsystem 100 and stored in the asset location database. It should beappreciated that the control system 100 can modify the projectioncoordinates based on the updated scanned position.

IV. EXEMPLARY CONTROL SYSTEM CONFIGURATION

In various embodiments, the control system 100 may comprise a pluralityof modules, each module configured to perform at least a portion of thefunctions associated with the methods described herein. For example, thecontrol system 100 may comprise an acquisition module, a locationmodule, and a notification module. Although described herein as beingindividual components of the control system 100, the various modules mayoperate on a combination of one or more devices (e.g., the user device110, the acquisition/display entity 804 (for capturing the asset 10information), the location device 415 (where provided), and/or thecontrol system 100), such that each device performs the functions of oneor more modules.

A. Acquisition Module

In various embodiments, the acquisition module may be configured toobtain asset identifier data associated with an asset 10 to be sortedand/or picked. This asset identifier data may be obtained, in part, viaan order placed by a customer desiring transport and delivery (e.g.,picking, as a first step) of the asset or package. In other embodiments,the asset identifier data may be obtained, in part, via theacquisition/display entity 804 associated with a conveyor belt of thelike, transporting packages or assets to a sort location.

In various embodiments, the asset identifier data may comprise a uniqueasset identifier such as a tracking number or code, and data definingthe one or more appropriate locations 400 for the asset 10 as it movesbetween an origin and a destination, and/or the like.

As a non-limiting example, the acquisition module may be configured toobtain data from the user device 110 (e.g., of FIGS. 3 and 4 ) and/orthe acquisition device 810 (e.g., of FIG. 12 ). In various embodiments,the data received from the user device 110 and/or the acquisition device810 may include the entirety of the asset identifier data and thereforethe acquisition module need only receive asset identifier data from oneof the user device 110 and/or the acquisition device 810. However, invarious embodiments, the data received from the user device 110 (FIGS. 3and 4 ) and/or the acquisition device 810 (FIG. 12 ) may comprise only aportion of the asset identifier data, and the acquisition module may beconfigured to obtain the remainder of the asset identifier data from oneor more other sources. As another non-limiting example, the acquisitionmodule may be configured to search one or more databases incommunication with the control system 100 for asset identifier datacorresponding to the data received from the user device 110 and/or theacquisition device 810. The acquisition module may additionally oralternatively be configured to receive and store at least a portion ofthe asset identifier data corresponding to the asset 10 that is storedin one or more databases.

In various embodiments, the acquisition module may be configured totransmit at least a portion of the asset identifier data to one or moredevices (e.g., the user device 110) and/or one or more modules (e.g.,the location module and/or the notification module). Moreover, uponreceiving the asset identifier data regarding an asset 10 to be sorted,the acquisition module may be configured to link or otherwise associatethe user device 110 and the asset identifier data. As will be describedin greater detail herein, the user device 110 may be associated with theasset identifier data by storing at least a portion of the assetidentifier data in a memory associated with the user device 110.

B. Location module

The location module may be configured to receive asset identifier datafrom the acquisition module. The sort location module is configured toascertain the appropriate location 400 and/or the appropriate positionwithin the location 400 for the asset 10 based at least in part on theasset identifier data. In certain embodiments, the location module maybe configured to determine the appropriate location 400 based at leastin part on the asset identifier data and location data that isassociated with the each of the plurality of locations 400. The locationdata may be generated based not only upon the asset identifier data, butalso upon the environmental mapping conducted via the user device, asdescribed elsewhere herein.

In various embodiments, each of the plurality of locations 400 may beidentified by location data, which may include a unique locationidentifier. The unique location identifier may comprise a uniquecharacter string individually identifying each of the plurality oflocations 400. In various embodiments, the location data may define anysubsequent processing to be performed on assets 10 within each location400 and/or 1400, and may comprise the unique sort location identifierfor each of the plurality of locations 400/1400 the assets 10 will passthrough. In various embodiments, the location module may determinewhether the processing to be performed on assets 10 in each of theplurality of locations 400 (as defined in the location data) will movethe asset 10 closer to its final destination.

In various embodiments, the location module may determine whether theprocessing steps to be performed on the assets 10 in each of thelocations 400/1400 complies with the service level (e.g., Same Dayshipping, Next Day Air, Second Day Air, 3 Day Select, Ground shipping,and/or the like) corresponding to the asset 10. As a non-limitingexample, the location module may determine the appropriate location foran asset 10 to be delivered to 123 Main Street, Atlanta, Ga. is adelivery vehicle that will deliver other assets 10 to the same addressor nearby addresses (e.g., along the same delivery route). As a secondnon-limiting example, the location module may determine the appropriatelocation for an asset 10 to be delivered to 345 Broad Street, LosAngeles, Calif. via Next Day Delivery is a pallet to be loaded onto aplane destined for Los Angeles, Calif. As yet another non-limitingexample, the location module may determine the appropriate location foran asset 10 prior to its fulfillment for delivery, which location may becharacterized—as done elsewhere herein—as a pick location for the asset.

After determining the appropriate location 400/1400 and/or theappropriate position for the asset 10 within the location 400/1400, thelocation module may be configured to transmit data defining theappropriate location 400/1400 and/or the appropriate position for theasset 10 within the location 400/1400 to one or more devices (e.g., theuser device 110) and/or modules (e.g., the notification module).Additional details in this respect are provided in U.S. Ser. No.15/390,109, the contents of which as are hereby incorporated byreference in their entirety.

C. Notification Module

In various embodiments, the notification module may receive dataindicating whether the location 400 and/or 1400 (e.g., as transmitted tothe control system 100 via the user device) is the appropriate sort orpick location (e.g., as determined by the control system 100) for theasset or package being handled. As described herein, the notificationmodule may cause one or more alerts to be generated in order to notifythe user 5 (e.g., sort or pick personnel, more generally the carrierpersonnel) whether the asset 10 should be deposited in the location 400and/or picked therefrom, however as the case may be. For example, thenotification module may be configured to transmit confirmation dataand/or mistake data to the user device 110 in order to cause the deviceto generate an alert discernible by the user 5 (e.g., carrier personnel)indicative of the appropriate sort location for the asset 10. Toascertain whether confirmation data and/or mistake data is appropriatefor transmission, the user device 110 (and/or sensors associatedtherewith, e.g., three-dimensional sensors) may be configured todetermine not only the position of the asset but also the position ofthe user's hands (e.g., including not only location, but also gestures),so as to gauge whether or not sorting and/or picking of the asset isproceeding properly. For example, the camera 116 may utilize objectrecognition algorithms that identify whenever a person is clasping anobject in a particular manner to determine properness.

In various embodiments, the notification module may cause the userdevice 110 to audibly provide the user with a confirmation message(e.g., via the speaker 117) upon a determination that the location400/1400 is the appropriate sort or pick location. In variousembodiments, the notification module may alternatively or additionallycause one or more sounds to be generated, one or more lights toilluminate, one or more mechanical assemblies to move, and/or otherprocesses discernible by a user 5 to operate and thus indicate to theuser 5 whether the location 400/1400 is the appropriate location. Itshould also be understood that notifications may be generated—andcommunicated to the user via the user device—not only when the user isat the location (e.g., for picking or sorting), but also during travelof the user to/from the location relative to other locations in thewarehouse or facility. As a non-limiting example, with reference to FIG.9 , as the user travels down the hallway or open path, were the user tonavigate contrary to the navigational projection 710 provided, the userdevice could generate an audible (or other type) notification to theuser—either independently, or upon cue received via the control system100. In certain embodiments, notifications may be generated andcommunicated (e.g., via the network or otherwise) to one or more partiesother than the user of the user device (e.g., carrier supervisorypersonnel, other internal carrier personnel (e.g., quality assurancerepresentatives, or the like), external personnel, external third partyentities, or the like). Any of the notifications and/or communicationsdescribed herein may be so communicated, whether to the user aloneand/or to parties other than the user and/or to a combination of both,as may be desirable.

Moreover, the notification module may be configured to generate an alertafter associating asset identifier data with location data and/or cueingan asset or package for picking. The notification module may beconfigured to generate an alert to inform the user 5 (e.g., carrierpersonnel) or other users regarding asset identifier data beingassociated with location data and/or the immediate need for navigationor travel to occur toward the location for picking of the asset orpackage or otherwise.

According to various embodiments, whether adjacent a location 400/1400or a conveying mechanism 802, the notification module may be configuredto generate one or more navigational projections (e.g., 710, 715, 1401and/or the like, with reference to FIGS. 8-11C by way of non-limitingexample) to convey navigational instructions to the user 5. It should beunderstood that according to various embodiments, the navigationalprojections may be computer-generated and/or overlaid over an augmentedreality environment, which may in certain embodiments be displayed tothe user via the utilized user devices 110. In at least the hands-freeembodiment, the navigational projections may be generated via thepivoting laser projector (e.g., 118 of FIG. 5 ) of the user device 110.FIGS. 8-11C and also FIGS. 13A-13F illustrate various types ofnavigational projections as may be generated via the notificationmodule, when conveyed via the control system 100 further to the userdevice 110. In at least one embodiment, it may be understood that thenavigational projections may be generated at/by the user device 110,independent of the control system 100, upon receipt from the controlsystem of only a new “pick” or “sort” command for a particular asset orpackage. In various embodiments, the text indicia and navigationalprojections of FIGS. 13A-13F occur via one or more components of thedevice component 114 (e.g., laser projector 118 to project the arrow1303 and the indicia 1309 of FIG. 13A).

FIG. 13A includes the environment 800 that the user is physicallylocated in, which includes the conveyance device 1305, an asset 1301,various location devices, one of which is indicated by the device 1307.As described above, the location devices help map the environment 800.The text indicia (which may also be considered a navigationalprojection) 1309 may be projected within the environment 800, whichcommands the user to “push forward” the asset 1301. The navigationalprojection 1303 (i.e., the arrow) also assists or illustrates thedirection in which the user should push the asset 1301 forward on theconveyance mechanism 1305. FIG. 13B includes the same environment 800,except a different asset 1315 arrives and the text indicia 1311 commandsthe user to “push” the asset 1315 to the “other side.” The navigationprojection 1313 helps guide the user to show which direction to push theasset 1315. In this way, these visual frames together help guide andinstruct the user for handling of assets. Turning to FIG. 13C, anotherasset 1317 arrives and the text indicia 1319 prompts the user to “pickand sort” the asset 1317 to pick up, sort, and place within a sortinglocation.

FIG. 13D illustrates prompting the user to sort the asset in aparticular location within the environment 1400. In some embodiments,the instructions illustrated in FIG. 13D occur in response to the userpicking up the asset 1317 as illustrated in FIG. 13C. The environment1400 includes the text indicia 1401 that states “look that way,” whichis accompanied by the navigational projection 1403 that illustrates whatdirection the user should walk in order to sort the asset. FIG. 13Eillustrates the correct location to sort an asset within the environment1400. In some embodiments, the instructions illustrated in FIG. 13Eoccur in response to the user moving responsive to the text indicia 1401and/or the navigational projection 1403 of FIG. 13D. The environment1400 includes the text indicia 1407 “sort here” indicating, along withthe navigation projection 1405, where the correct cell or location forsorting the asset is. FIG. 13F illustrates the combined environments 800and 1400 of FIGS. 13A through 13E. Accordingly, the user 1411 picks upthe asset 1413 responsive to viewing a first set of navigationalprojections and/or text indicia and places the asset 1413 within thecorrect location responsive to viewing a second set of navigationalprojections and/or text indicia.

V. EXEMPLARY SYSTEM OPERATION

A. Exemplary User Device Operation

FIGS. 8-13F illustrate an exemplary environment in which assets 10 aremoved amongst various locations 400, which locations may be picklocations (e.g., for storage of an asset in a warehouse or the like;e.g., FIGS. 8-11C in particular), a conveyor belt location (e.g., FIG.12 in particular), and/or sort locations (e.g., for placement of anasset following distribution from a pick location to a conveyor beltlocation; e.g., FIGS. 13A-F in particular). In various embodiments, auser 5 (e.g., sort personnel) may utilize a user device 110 as describedherein while transporting assets 10. As described herein, the userdevice 110 may be configured for receiving information regarding aparticular asset 10 to be transported, whether from the control system100 or otherwise, for guiding the user 5 to a location in which theasset 10 is or should be transported to, and for informing the user 5whether the asset 10 is being located (e.g., via navigationalprojections) and/or transported appropriately.

FIG. 6 illustrates exemplary steps carried out by the user device 110according to various embodiments of the present disclosure to achievethe advantages and capabilities outlined above. In Block 501, aninitialization or calibration of the user device 110 may be conductedaccording to various embodiments. In certain embodiments this step orblock may be optional; in other embodiments, it need only be conductedperiodically, for example upon initial use of the user device and/orupon receipt—from the control system 100—of a notification that anenvironment in which the user device operates has been altered orupdated. As described herein above, per block 501 an environment that auser is located in is mapped based at least in part on generating amultidimensional (e.g., 3-D) graphical representation of theenvironment.

According to various embodiments, with reference to FIG. 6 and FIG. 8 ,during Block 501 of FIG. 6 , the user device 110 may be worn by a user 5so as to map an environment 700 in which the user device 110 is to beused. A graphical representation 701 of the environment may be generatedand/or stored via the user device 110; whereby storing may occur locallyat the user device and/or be stored at and/or synced with the controlsystem 100, for example, upon completion of the mapping procedure. Asmay be understood from FIG. 8 and in some embodiments, the mappingprocedure of step or Block 501 involves the user, while wearing the userdevice 110, to move through the environment 700, which movementnecessarily involves passage of locations 400/1400 (e.g., shelves) andvarious assets or packages 10. During the mapping procedure,three-dimensional depth sensors 119 (e.g., the depth sensors 119 of FIG.5 ) of the user device may be utilized to capture the data required togenerate the graphical representation 701. In conjunction with a camera(e.g., the camera 116 of FIG. 5 ), locations of shelving (e.g.,locations 400) and also locations of assets/packages—for example in“pick locations”—may also be established, determined, and/or otherwisesaved at or by the user device 110. Movement of the user 5 through theenvironment 700 defines progressive mapping zones 705, through which thethree dimensional depth sensors are configured to scan during the courseof mapping. To facilitate and further optimize the mapping procedure, incertain embodiments, one or more commands may be transmitted to the user(e.g., via the speaker 117) from the control system (or otherwise), forexample to instruct the user to “turn left” at specific intervalsconfigured to ensure that the entirety of the environment 700 (or adesired portion thereof) is sufficiently mapped. This feature will, forexample, minimize and/or eliminate instances of users not covering everyarea within the environment for which mapping may be desired and/orrequired. In some embodiments, this mapping using 3D scanningcapabilities to detect the immediate environment of the user projectionsare modified to avoid visual distortion. Accordingly, shapes and imagesprojected are adjusted to the shape of the surface they will beprojected upon in particular embodiments. In these embodiments, objectrecognition devices and/or scanners, such as cameras, located within adevice component (e.g., device component 114) can identify the contoursof the environment. In response, a projection can be made in theenvironment based on the shapes (e.g., uneven surfaces) of theenvironment.

Returning to FIG. 6 , in Block (or step) 502, the user device 110 isconfigured to receive and/or associate product (e.g., package or asset)locations within the mapped environment. This may be via utilization ofidentifiers 415 (as described elsewhere herein), via transmission ofasset location information from the control system to the user device,and/or the like.

According to various embodiments, upon completion of the mapping of theenvironment 700 and the association of product (e.g., package or asset10) locations therein, the user device 110 is calibrated for operationalmode or use, which use may occur in either (or both) a pick and a sortmode. In the pick mode, the user device is configured to guide a userthereof to a location in which a package or asset 10 may be picked or“pulled” for fulfillment of an order (e.g., within the environment 800of FIG. 13A); in the sort mode, the user device guides the user (e.g.,from a conveyor belt to a sort location), defined to enable furthertransport and handling of the asset or package in route to a customer orthe like (e.g., within the environment 1400 of FIG. 13D).

If it is determined in block 515 that pick mode is appropriate, the userdevice 110 proceeds to step or Block 503, wherein pick location data isreceived. In certain embodiments, the pick location data is received—atthe user device 110—from the control system 100, for example, uponreceipt—at the control system—of a customer order for a particular asset10 or package. Based upon the received pick location data in Block 503,the user device 110 is configured to, in certain embodiments, generatepick instructions in Block 504. The generation of pick instructions inBlock 504 may entail compilation of a route through which the user ofthe user device 110 must travel—from their present location—so as toreach the location of the asset needing to be picked. Block 504 mayfurther entail generation of a plurality of potential navigationalprojections (e.g., as described in FIGS. 9-11C) that will be necessaryto accurately and efficiently guide or direct the user of the userdevice to the pick location. In Block 504, in certain embodiments,multiple possible routes may be determined and assessed, with either theuser device (automatically) or the user (via an interface selection)choosing an optimal route, for which the navigational projections maythereafter be established. Associated audible commands may also begenerated/established, in conjunction with the navigational projections,should it be desirable to—in addition or alternatively to thenavigational projections—also audibly guide (e.g., via the speaker 117)the user, instructing them to, for example, “turn left after the nextshelving row, as depicted in FIG. 9 , by way of non-limiting example.

Upon completion of step or Block 504 the user device 110 is configuredto proceed to Block 505, wherein the navigational projections and/oraudible instructions are dynamically displayed and/or otherwise providedto the user of the user device, characterized generically as “pickinstructions.” It should be understood that, according to certainembodiments, Blocks 504 and 505 need not be separate and distinct stepsor blocks; instead, as will be described below, as the user movesthrough the environment 700, the user device 110 may be configured todynamically generate and display various navigational projections and/oraudible instructions. In at least those embodiments, Block 504 mayentail merely identifying—at the user device 110—the user's presentlocation, the pick location, and a variety of pathways or routesthere-between.

Reference is made now to FIGS. 9-11C, which depict a variety ofexemplary navigational projections that may be generated and displayedvia the user device 110 as a user 5 wearing the same moves around anenvironment 700. As may be understood by contrasting FIG. 9 with FIGS.10-11C, ahead of the user arriving proximate the pick location, variousnavigational projections 710 may be generated and displayed (e.g., viathe pivoting laser projector 118 of FIG. 5 ). These navigationalprojections 710 may be two or three-dimensional in nature and—asgenerally understood—provide directional guidance to the user, as towhich direction they should move or turn. As illustrated, by way ofnon-limiting example, in FIG. 9 , the navigational projection 710 incertain embodiments may be a two-dimensional directional arrowconfigured to instruct a user wearing a user device to make a changerelative to their present movement pattern. In other embodiments, suchas those illustrated in FIGS. 12-13F, the navigational projections,therein described as indicators 810 or navigational projections 810, maybe three-dimensional in nature.

Returning now to FIGS. 9-11C, with reference now in particular to FIGS.10-11C, therein may be seen certain navigational projections 715, 715A,715B, 715C, which may be generated and displayed for the user—via theworn user-device 110—as they approach the proximity of the picklocation. Proximity may be defined as within a specific row upon whichthe asset or package to be picked is located. As may be understood fromFIGS. 11A-11C in particular, the navigational projections 715 mayinclude a frame and/or a checkmark, showing the right location to pickthe asset or package from. In certain embodiments, text indicating aquantity may also be illustrated, as in FIG. 11A via projection 715A,which projection encompasses multiple package or asset boundaries, asmay be recognized and detected by the user device 110 upon approach tothe asset or package. The boundaries may, as a non-limiting example, bedetermined by the user device 110, at least in part, based upon assetinformation received from the control system 100. In other embodimentssoftware embedded upon the user device 110, in conjunction with thecamera (e.g., the camera 116 of FIG. 5 ) may be configured to, via aniterative machine learning-type algorithm and/or object recognitionalgorithm, recognize and determine asset/package dimensions and thusboundaries over a period of time.

FIG. 11A illustrates one exemplary embodiment, in which, as alluded toabove, the navigational projection 715A encompasses multiple assets orpackages 10 in a particular location 400, with textual instructions alsobe generated and provided to the user to “pick 2” of the highlighted orframed packages. In other embodiments (not illustrated), theinstructions may say “pick these two” assets, with the frameencompassing only two specific assets on the shelf or location 400. Instill other embodiments (e.g., FIG. 11C), multiple frames andinstructions associated with navigational projection 715C may beprovided, distinctly identifying three assets or packages that need tobe picked. A single asset pick embodiment is also illustrated in FIG.11B, whereby a navigational projection 715B is provided, similar to theprojection 715C, in that each frame and textual instruction generatedand displayed surrounds and is overlaid relative to a single package orasset 10. It should be understood that the illustrations and embodimentsof FIGS. 11A-C are non-limiting in nature; additional and/oralternatively configured navigational projections 715 may be envisionedwithin the scope of the inventive concept described herein.

Returning now to FIG. 6 and remaining with Block 505, in addition toaudible guidance instructions that may be provided/generated inconjunction with the navigational projections 710, 715 of FIGS. 10-11C,various embodiments may involve the user device 110 further generatingand transmitting to the user 5 audible alerts when deviations occur. Forexample, the user device 110—via its speaker 117 of FIG. 5 —may providegeneral feedback to the user, such as “you picked the wrong asset;please await further instructions” or “you turned the wrong way; pleasestand still pending updated navigational projections becoming visible.”Alternative or additional “feedback” type alerts and/or instructions maybe generated, in a near real time or real time manner, based upon theuser's responsiveness (and accuracy of movement) relative to theprovided navigational projections.

Turning now to step or Block 506 in FIG. 6 , upon the user with the userdevice 110 reaching the pick location, the user device 110 is configuredto further detect asset handling, specifically when the asset or packagehas been picked up by the user. In certain embodiments, feedback mayalso be provided at this junction, via the speaker 117 of the userdevice 110. In these and still other embodiments, the user device 110may capture an image of the “picking” (e.g., via its camera 116) and/ortransmit the same to the control system 100 for centralized/remoteverification of picking accuracy and completeness. This action may occuralso in conjunction with Block 507, whereby one or more pick-relatednotifications may be generated and/or transmitted by the user device110, whether to the user 5 and/or the control system 100.

According to various embodiments, the detection of the “picking” may beconducted by the user device 110 via a collision detection algorithm, asdetailed elsewhere herein. As generally understood, though, suchalgorithm(s) are configured to detect changes in movement relative tothe user device 110, whereby if an item or person (e.g., a user)associated with or wearing the user device 110 encounters acollision—for example by picking up and physically touching an asset orpackage—that “collision” likewise registers at the user device. In thismanner, the user device 100 may be programmed to transition from aguidance mode to—at least temporarily—a report or notification mode, soas to convey—for example to the control system 100—that the pick hasoccurred.

In certain embodiments multiple algorithms may be utilized. One may beto identify what an asset or package is, namely what its physicalboundaries entail. Another is to interpret when a user's hands (or thelike) collide with and pick up (or set down) the asset or package. Eachmay be assisted, not only via the depth sensors 119 of the user device,but also the camera 116 thereof. In certain embodiments, at least twoand in some instances four depth sensors 119 may be provided, each witha working range between 0.85 and 3.1 meters (alternative ranges may alsobe provided). Using data collected via the depth sensors, theenvironment may thus not only be mapped, but changes therein, includingcollisions between objects—including packages, assets, users, and/ordevices such as forklifts operated by a user—may be detected andaccounted for. In some embodiments, particular algorithms able toidentify the parcel itself using machine learning techniques to searchfor physical clues (e.g., size, color, scratches, or any feature, evenmicroscopic that may lead to uniquely id the parcel) without needing toread any parcel id label or barcode.

Remaining with FIG. 6 and returning to Block 515, in certain instancesthe user device 110—whether independently or following a successful pickprocedure—may be configured to operate in a sort mode, which correspondsgenerally to utilization of the user device to move assets or packagesto/from various sorting locations within a mapped environment, ascontrasted with a user locating the asset or package from a stored“pick” location. The sequence in sort mode initiates with the userdevice 110, in Block 508, detecting handling of an asset by the user(e.g., via the collision algorithm described previously herein) and/orobtaining asset identifier data from the asset or package (e.g., via thecamera 116, whether independently at the user device 110 or further inconjunction with an exchange of certain asset/package data with thecontrol system 100).

Upon obtaining the asset identifier data, the user device 110 is able todetermine and/or receive sort location data for the asset or package 10in Block 509. Based thereon, much like in Blocks 504-505 and 507 (in thecontext of sorting), the user device 110 is configured to—according tovarious embodiments—generate sort instructions in Block 510, dynamicallydisplay sort instructions (e.g., navigational projections, text indicia,and/or audible instructions and the like) in Block 511, andgenerate/transmit one or more sort-related notifications in Block 512.In some embodiments, the generating and displaying of one or morenavigational projections configured to guide the user to an assetlocation within an environment is based at least on associating one ormore asset locations within a mapped environment. In variousembodiments, the displaying occurs within an environment that a user isin without regard to a necessary medium (e.g., lens, projector screen,etc.). In these embodiments, the projection is displayed in open spacewithin the environment. It should be understood that any of Blocks509-512 may be substantially the same or identical (the same) as thosein Blocks 503-505 and 507, as previously detailed herein; in certainembodiments, though, one or more of the Blocks may be configureddifferently for sort versus picking mode.

Additional details relative to the utilization of the user device 110 insort mode may be understood with reference to FIGS. 13A-F, which figuresare also described elsewhere herein. Additional details regardingsorting procedures may also be understood with reference to U.S. Ser.No. 15/390,109, the contents of which as are hereby incorporated byreference in their entirety.

B. Exemplary Control System Operation

FIG. 7 illustrates exemplary steps carried out by the control system 100according to various embodiments of the present disclosure. Asillustrated in FIG. 7 , the control system 100 may receive assetidentifier data at Block 601. As indicated herein, the asset indicatordata may be received from the user device 110, the acquisition device810, and/or the one or more location devices 415 at a location 400.Further details regarding the scope and contents of the asset identifierdata have been described previously herein. Still additional details inthis respect may be understood with reference to U.S. Ser. No.15/390,109, the contents of which as are hereby incorporated byreference in their entirety.

At Block 602, the control system 100 may be configured to determine theappropriate location 400 for the asset 10 and/or the appropriateposition within the location for the asset 10. In various embodiments,the determination of the appropriate location for the asset 10 may bebased at least in part on the received asset identifier data. Moreover,the control system 100 may utilize location data corresponding to eachof the locations 400 to determine whether any subsequent processing tobe performed on assets 10 at each location 400 will move the asset 10closer to its final destination. As a non-limiting example, the controlsystem 100 may determine the appropriate location for an asset 10 to bedelivered to 123 Main Street, Atlanta, Ga. is the delivery vehicle thatwill deliver other assets 10 to 123 Main Street, Atlanta, Ga. Additionaldetails in this respect may be understood with reference to U.S. Ser.No. 15/390,109, the contents of which as are hereby incorporated byreference in their entirety.

Referring again to FIG. 7 , at Block 703 the control system 100 may beconfigured to transmit data identifying the appropriate sort location tothe user device 110. As noted herein, the user device 110 may beconfigured to generate an indicator (e.g., visual indicators ornavigational projections 710/715/810) discernible by the user 5 (e.g.,carrier personnel) regarding the appropriate pick or sort location forthe asset 10. However, as noted herein, each asset 10 may haveinformation indicative of an appropriate location printed thereon, andaccordingly the control system 100 may not need to—in thoseembodiments—transmit appropriate location data to the user device 110.

The control system 100 may also be configured to receive a variety ofdata—including location data—from the user device 110 at Block 604. AtBlock 605, the control system 100 may subsequently compare theappropriate location (at which the user for picking or the asset forsorting should be located) and the actual location data received atBlock 604 to determine whether the user device 110 is proximate theappropriate location. As indicated at Block 606, the remaining steps tobe completed may be selected based at least in part on a determinationof whether the location is an appropriate (or desired/accurate)location. Additional details in this respect may be understood withreference to U.S. Ser. No. 15/390,109, the contents of which as arehereby incorporated by reference in their entirety.

Upon a determination that the user device 110 is proximate an incorrectlocation 400, the control system 100 may generate mistake data at Block610. Upon generating the mistake data, the control system 100 maytransmit the mistake data to the user device 110 at Block 611. Asindicated herein, the user device 110 may be configured to generate amessage discernible by the user 5 (e.g., carrier personnel) indicatingthe user device 110 is proximate an incorrect location 400 (e.g., asillustrated in FIG. 13D). In various embodiments, the control system 100may be configured to associate the asset identifier data with thelocation data corresponding to the location 400 at Block 612. At Block613, the user 5 may continue transporting the asset 10 (and consequentlythe user device 110) to another (ideally correct) location 400. Theprocess may return to Block 601 in such scenarios and repeat the recitedsteps.

Referring again to Block 606, the process may proceed after comparingthe actual/received location data and the appropriate location data forthe asset 10 (illustrated as Block 605) with reference to Blocks 607-609if the user 5 approaches the appropriate location. In the context ofsorting procedures, additional details in this respect may be understoodwith reference to U.S. Ser. No. 15/390,109, the contents of which as arehereby incorporated by reference in their entirety.

The control system 100 may be further configured to generate one or morealerts regarding the association between the asset identifier data andthe location data. The control system 100 may be configured to generatean alert to inform the user 5 (e.g., carrier personnel) or other usersregarding asset identifier data being associated with location data.

VI. CONCLUSION

Many modifications and other embodiments of the disclosures set forthherein will come to mind to one skilled in the art to which thesedisclosures pertain having the benefit of the teachings presented in theforegoing descriptions and the associated drawings. Therefore, it is tobe understood that the disclosures are not to be limited to the specificembodiments disclosed and that modifications and other embodiments areintended to be included within the scope of the appended claims.Although specific terms are employed herein, they are used in a genericand descriptive sense only and not for purposes of limitation.

What is claimed is:
 1. A system comprising: one or more processors; andone or more computer-storage media having one or more instructionsstored that, when used by the one or more processors, cause the one ormore processors to perform operations comprising: initializing ascanning device secured to a storage area of a delivery vehicle;calibrating a projection device also secured to the storage area of thedelivery vehicle, wherein the projection device comprises a light sourcethat is rotatable at least one of horizontally or vertically by one ormore stepper motors; obtaining, via a user device, a present location ofthe user device; determining, via scanning information obtained from thescanning device, a storage location and an asset identifier of an assetafter the asset is placed within the storage area, wherein determiningthe storage location of the asset is based at least in part on a firstset of visual characteristics included in the scanning information onthe asset and determining the asset identifier is based at least in parton a second set of visual characteristics includes in the scanninginformation on a label on the asset that is associated with a defineddelivery destination; obtaining a current location of the deliveryvehicle based at least in part on detected location data; determiningthat the current location is within a threshold distance of the defineddelivery destination; determining that one or more predefined conditionsare satisfied; and based at least in part on determining that thecurrent location is within the threshold distance of the defineddelivery destination and the one or more predefined conditions aresatisfied, causing the one or more stepper motors to rotate the lightsource at least one of horizontally or vertically to emit a navigationalprojection that corresponds to a route to the storage location of theasset, wherein the route is determined based at least in part on thepresent location and the storage location.
 2. The system of claim 1,wherein the navigational projection comprises at least one of a twodimensional projection, a three dimensional projection, or a holographicimage.
 3. The system of claim 1, wherein the operations further compriseexchanging, with the user device, asset-related data related to ahandling of the asset.
 4. The system of claim 1, wherein thenavigational projection is altered to indicate that the storage locationof the asset is behind another asset.
 5. The system of claim 1, whereinthe operations further comprise generating an audible instructionconcurrent with the navigational projection.
 6. The system of claim 1,wherein the one or more predefined conditions comprise at least one ofan indication that the delivery vehicle is in park or an indication thata door of the delivery vehicle has been opened.
 7. Acomputer-implemented method comprising: calibrating, by a computingdevice, a projection device secured to a storage area of a deliveryvehicle, wherein the projection device comprises a light source that isrotatable at least one of horizontally or vertically by one or morestepper motors; obtaining, by the computing device and from a userdevice, a present location of the user device; obtaining, by thecomputing device, an asset identifier for an asset located within thestorage area based at least in part on scanning information receivedfrom a scanning device secured to the storage area of the deliveryvehicle, wherein the asset identifier is obtained from a first set ofvisual characteristics included in the scanning information on a labelon the asset and is associated with a delivery destination defined forthe asset; generating, by the computing device, asset location data thatis associated with the asset identifier, wherein the asset location datadefines a position of the asset within the storage area based at leastin part on second set of visual characteristics included in the scanninginformation on the; determining, by the computing device and based atleast in part on location data received from one or more locationsensors, that a current location of the delivery vehicle is within athreshold distance of the delivery destination; determining, by thecomputing device, that a predefined condition has been satisfied; andresponsive to the currently location of the delivery vehicle is withinthe threshold distance of the delivery destination and the predefinedcondition is satisfied, causing, by the computing device, the one ormore stepper motors to rotate the light source at least one ofhorizontally or vertically to emit a navigational projection thatcorresponds to the position of the asset within the storage area,wherein the navigational projection corresponds to a route to theposition of the asset, and the route is determined based at least inpart on the present location and the asset location data.
 8. Thecomputer-implemented method of claim 7, wherein the asset location datacomprises coordinates for the asset within the storage area, and thecoordinates are generated based at least in part on a location of thescanning device.
 9. The computer-implemented method of claim 7, whereinthe scanning information further comprises a scanned position of theasset within the storage area, and the method further comprises:modifying, by the computing device, projection coordinates of theprojection device based at least in part on the scanned position anddetermining that the current location is within the threshold distanceof the delivery destination, wherein the navigational projection isemitted based at least in part on the projection coordinates.
 10. Thecomputer-implemented method of claim 9 further comprises receiving, bythe computing device, additional scanning information comprising anupdated scanned position of the asset within the storage area; andmodifying, by the computing device, the projection coordinates based atleast in part on the updated scanned position.
 11. Thecomputer-implemented method of claim 7, wherein the navigationalprojection comprises at least one of a two dimensional projection, athree dimensional projection or a holographic image.
 12. Thecomputer-implemented method of claim 7, further comprising exchanging,with the user device, asset-related data related to a handling of theasset.
 13. The computer-implemented method of claim 7, wherein thenavigational projection is altered to indicate that the asset is locatedbehind another asset.
 14. One or more non-transitory computer-storagemedia having computer-executable instructions embodied thereon that,when executed by a computing device, perform operations comprising:calibrating a projection device secured to a storage area of a deliveryvehicle, wherein the projection device comprises a light source that isrotatable at least one of horizontally or vertically by one or morestepper motors; determining a physical location of the delivery vehiclebased at least in part on detected location data; obtaining, from a userdevice, a present location of the user device; obtaining an assetidentifier for an asset stored within the storage area of the deliveryvehicle based at least in part on visual scanning information receivedfrom a scanning device secured to the storage area of the deliveryvehicle, wherein the asset identifier is obtained from a first set ofvisual characteristics included in the visual scanning information on alabel on the asset and is associated with a defined deliverydestination; determining a storage location within the storage area ofthe delivery vehicle based at least in part on a second set of visualcharacteristics included in the visual scanning information on theasset; determining that the physical location of the delivery vehicle iswithin a threshold distance of the defined delivery destination;determining one or more predefined conditions are satisfied; and causingthe one or more stepper motors to rotate the light source at least oneof horizontally or vertically to emit a navigational projection directedto the storage location based at least in part on determining that thephysical location of the delivery vehicle is within a threshold distanceof the defined delivery destination and the one or more predefinedconditions are satisfied, wherein the navigational projectioncorresponds to a route to the storage location of the asset, and theroute is determined based at least in part on the present location andthe storage location.
 15. The one or more non-transitorycomputer-storage claim 14, wherein the visual scanning informationfurther comprises a visually determined position of the asset within thestorage area, and the operations further comprise: modifying projectioncoordinates of the projection device based at least in part on thevisually determined position and determining that the physical locationis within the threshold distance of the defined delivery destination,wherein the navigational projection is emitted at least in part based onthe projection coordinates.
 16. The one or more non-transitorycomputer-storage claim 15, wherein the operations further comprise:obtaining additional visually determined information including anupdated visually determined position of the asset within the storagearea; and modifying the projection coordinates based at least in part onthe updated visually determined position.
 17. The one or morenon-transitory computer-storage media of claim 14, wherein theoperations further comprise exchanging, with the user device,asset-related data related to a handling of the asset.
 18. The one ormore non-transitory computer-storage media of claim 15, wherein thedetected location data is generated based at least in part on positiondata of the scanning device.